Boosting Knowledge Graph Foundation Models via Enhanced Negative Sampling

Knowledge graphs (KGs) are the core backbone of numerous downstream tasks such as question answering and recommender systems. However, despite all this, KGs are often very incomplete. To perform zero-shot link prediction in unseen KGs, which have different entity/relation vocabularies from those used for pre-training, KG foundation models (KGFMs) receive a wide range of attention. Existing KGFMs often perform training using random negatives, which are constructed by replacing the head or tail entity of a positive triple with a random entity. However, these negatives are often constructed with limited quality, providing weak supervision for KGFM training. In this paper, we propose a simple yet effective adaptive negative sampling approach, KMAS, to enhance existing KGFMs. KMAS constructs hard negative triples through the updated relation embeddings generated from the existing KGFM's relation encoder. To further adaptively align with the KGFM’s evolving capability to distinguish between positive and negative samples during the training process, KMAS adjusts the ratio of hard negative triples dynamically throughout the whole training process: after a warmup phase, it increases the ratio linearly and then decreases linearly. Extensive experiments are conducted over 44 data sets. Experimental results demonstrate that KMAS can enhance many SOTA KGFMs without requiring excessive additional memory consumption, and it incurs no additional time cost on most KGFMs.