- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper argues that existing knowledge graph embedding (KGE) models treat each embedding's absolute coordinates as the "semantic meaning" of an entity or relation, and proposes DiffE as an alternative. In DiffE, each entity (relation) embedding is replaced by the mean, over all other entities (relations), of its embedding differences passed through a one-layer feed-forward layer and a tanh. The resulting representations are fed into the unchanged scoring function of an existing KGE model (TransE, DistMult, RotatE, HousE, GoldE). Experiments on WN18RR, FB15k-237 and YAGO3-10 report gains of roughly one MRR point over each base model.
Critical strengths
The MRR improvement from DiffE over the base models is consistent; it holds for every base model and dataset in Table 2.
Critical weaknesses
-
The contribution is overstated. DiffE averages over all entities (or relations). The paper presents this as a change of paradigm in how semantic meaning is represented, but it is an aggregation layer on top of the same learned embeddings. Every entity's representation is a mean over the same full set of entities; this can be seen as a single step of message passing performed globally: each entity receives the messages from every other entity and aggregates them with a mean, as if on a complete graph, independently of the edges of the knowledge graph. Also note that every representation is still built from a learned per-entity embedding table, so DiffE cannot handle entities that were not seen during training, nor new graphs. "Rethinking identification" does not change what the model can or cannot represent.
-
Message-passing link predictors are missing from the comparison. Link prediction is the only task evaluated, so the method should be compared with the strongest link predictors, not only with other KGE models, e.g., message passing methods like NBFNet [1] is neither cited nor compared against. Its published results on the same benchmarks (MRR 0.415 on FB15k-237, 0.551 on WN18RR) are higher than every row of Table 2, including GoldE+DiffE (0.380 and 0.537). This is critical: since DiffE is a form of message passing (Weakness 1), it is not surprising that it improves over the base models, but message passing performed along the structure of the graph is known to work better. NBFNet propagates messages along the edges of the knowledge graph, over multiple layers and conditioned on the query, and obtains much larger gains while also adding inductive capabilities.
[1] Zhu, Zhaocheng, et al. "Neural bellman-ford networks: A general graph neural network framework for link prediction." Advances in neural information processing systems 34 (2021): 29476-29490.
Questions for authors
See weaknesses