HAVE WE SEEN THIS BEFORE? EVALUATING AND IMPROVING INCREMENTAL KNOWLEDGE GRAPH CONSTRUCTION

Knowledge graphs organize information from multiple sources for querying and reasoning. As new text arrives, maintaining a coherent graph requires integrating equivalent information without duplicating it and preserving distinct information without incorrectly merging it. Evaluating these complementary requirements is challenging because the appropriate update depends on what the graph already contains. We introduce DéjàKG, a controlled benchmark spanning initial Construction, Consolidation of equivalent information, and Extension with distinct new information. Single-fact inputs and predefined identity relationships specify expected graph updates, enabling deterministic, case-wise scoring without an additional large language model (LLM) judge. To address these integration challenges, we introduce IDEM, a framework that determines whether incoming entities and facts match existing knowledge or should be added to the graph. It compares their surface forms and descriptions using vector similarity, LLM-based decisions, or a hybrid of the two.