acceptodds
Under review as a conference paper at ICLR 2027

Two Negatives Make a Positive: Dual Negative Learning for Continual Knowledge Graph Embedding

Abstract

Continual Knowledge Graph Embedding (CKGE) aims to efficiently update Knowledge Graph (KG) representations to accommodate emerging facts while preserving previously learned knowledge. As new facts arrive, evolving graph structures and interactions between new and existing knowledge complicate continual adaptation. Existing CKGE methods primarily address these challenges by adapting representations, while the learning signals conveyed by negative samples remain insufficiently explored. Negative supervision in CKGE changes in two respects: accumulated facts alter the validity of negative samples, while model evolution affects their informativeness for training. To address these challenges, we propose EvoNS (Evolution-Aware Negative Supervision), a framework that jointly adapts negative construction and utilization through dual negative learning. Specifically, EvoNS exploits accumulated observations to construct history-consistent negatives. It further reweights individual negatives according to their model-perceived difficulty, enabling more informative negative signals to guide continual optimization. Extensive experiments on seven CKGE benchmarks demonstrate that EvoNS consistently outperforms existing baselines, achieving up to 12.40% relative improvement in MRR and reducing training time by up to 18.92% in several settings. The results demonstrate the effectiveness and efficiency of EvoNS for CKGE.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Continual Knowledge Graph Embedding, Evolving Knowledge Graphs, Adaptive Supervision

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_R725OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies negative sampling for continual knowledge graph embedding and proposes EvoNS, which consists of two components:

  1. HCNS: It filters negative samples that conflict with previously observed facts using accumulated historical knowledge and relation-specific type constraints.
  2. DANR: It adaptively assigns greater weight to difficult negatives based on the current model’s scores and applies a separate ranking loss to each positive-negative pair.

Experiments on seven continual knowledge graph benchmarks show improvements in MRR, Hits@K, and training efficiency over existing continual learning methods.

Critical strengths
  1. The paper addresses a meaningful negative-sample collision problem with clear motivation.
  2. The method is simple and can be readily incorporated into continual knowledge graph embedding frameworks.
  3. The evaluation covers seven datasets and includes predictive performance, efficiency, ablation, and sensitivity analyses.
  4. Both HCNS and DANR provide consistent improvements, with DANR making the larger contribution.
Critical weaknesses
  1. The methodological novelty is limited. HCNS is essentially filtered negative sampling over accumulated historical facts combined with relation-specific type constraints. DANR is closely related to existing self-adversarial negative sampling, hard-negative weighting, and per-sample margin-ranking losses. The main contribution therefore appears to be applying and combining existing ideas in the continual knowledge graph setting, rather than introducing a fundamentally new negative learning mechanism.
  2. The most relevant negative-sampling baselines are missing. The experiments compare EvoNS primarily with continual learning methods, but do not directly compare it with strong negative-sampling approaches—such as RotatE-style self-adversarial sampling, KBGAN, NSCaching, and type-constrained sampling—under the same TransE/CKGE framework. Since the ablation study indicates that most of the improvement comes from DANR, these comparisons are essential for establishing both novelty and effectiveness.
  3. Statistical significance is not reported. On some datasets, the absolute MRR improvements are relatively small, approximately 0.007–0.011. The paper does not report results over multiple random seeds, standard deviations, or significance tests. Given the inherent randomness of negative sampling, single-run results are insufficient to demonstrate that these improvements are reliable.
  4. The practical contribution of HCNS appears modest. The ablation results show that removing HCNS causes a substantially smaller degradation than removing DANR. Moreover, checking whether a candidate negative conflicts with previously observed facts is a natural filtering operation. The paper should more clearly explain the substantive difference between HCNS and standard filtered sampling.
Questions for authors
  1. How does EvoNS compare with standard self-adversarial negative sampling under the same TransE and continual learning framework?
  2. What is the performance when using only cumulative filtered sampling, without relation-specific type constraints?
  3. Are the reported results based on multiple random runs? If so, please report the means and standard deviations.
  4. Can EvoNS be integrated with methods such as STARK and FastKGE, instead of being evaluated only as a separate approach?
  5. What are the memory costs of the historical fact index and the relation-specific lookup tables used by HCNS?
Reviewer_VZS8OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EvoNS, an evolution-aware negative supervision framework for Continual Knowledge Graph Embedding (CKGE). The method addresses two challenges in negative learning: avoiding negative samples that conflict with previously observed facts and emphasizing informative negatives as the embedding model evolves.

EvoNS consists of two components. History-Consistent Negative Sampling (HCNS) combines global and relation-aware candidate generation with filtering against cumulative training facts. Difficulty-Aware Negative Reweighting (DANR) applies model-dependent weights to individual margin-ranking losses, emphasizing negatives that are difficult for the current model.

The authors evaluate EvoNS on seven CKGE benchmarks and report improvements in link prediction over several continual learning and CKGE baselines. The paper also includes module-level ablations, hyperparameter sensitivity studies, snapshot-wise results, and convergence analyses.

Critical strengths
  1. A relevant problem formulation for continual knowledge graph learning. The paper highlights two practical issues in CKGE negative supervision: conflicts with accumulated observations and the potential masking of individual ranking violations by group-level loss aggregation. These issues provide a reasonable motivation for revisiting negative construction and optimization in continual settings.

  2. Relatively comprehensive empirical evaluation. The experiments cover seven benchmark configurations with different graph-growth patterns. The authors also provide module-level ablations, sensitivity analyses, and snapshot-wise results, which help assess the contributions and behavior of the proposed framework.

Critical weaknesses

Recent KGE studies, such as TaKE (A type-augmented knowledge graph embedding framework for knowledge graph completion) , have already explored relation-aware or domain/range-constrained negative sampling to construct more informative negative triples. In particular, TaKE also considers relation-associated candidate entity pools and combines different sources of negative samples, which is conceptually similar to the candidate construction strategy used in HCNS.Although EvoNS extends this idea to continual knowledge graph embedding by incorporating cumulative historical consistency checking, the methodological distinction from these existing approaches is not sufficiently discussed.

Questions for authors

Could the authors clarify the key methodological differences between HCNS and recent relation-aware or type-constrained negative sampling approaches, particularly TaKE (2023) ? Beyond cumulative historical consistency checking, what are the distinctive contributions of HCNS in the continual learning setting?

Reviewer_MBSwOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper states that CKGE methods have neglected the negative aspect of training and introduces EvoNS, which consists of two components. HCNS combines uniform corruption with corruption taken from the entities that have been associated with a given relation, and it resamples any negative example that matches a cumulative training fact. DANR calculates a hinge for each negative and then weights these hinges using a softmax function based on the stop-gradient negative distances, with a temperature parameter τ. The base model used is TransE, with the historical parameters frozen and low-rank parameters being used for new entities and relations. When tested on seven FB15k-237 benchmarks, EvoNS outperforms nine baseline methods on all the ranking metrics that are reported, achieving relative MRR improvements ranging from 3.13% to 12.40%.

Critical strengths
  1. The diagnostic in Figure 1c/d is useful. If you hinge the averaged negative distance, the group loss is zero for most audited groups (roughly 60 to 75% in Figure 1d) even though individual margins are still violated. It's a concrete failure that's easy to check, and worth pointing out to anyone working with CKGE code.

  2. The paper is clearly written. The algorithms in Appendix E are complete enough to reimplement, and Appendix F is honest about what HCNS can't do: unobserved true facts can still end up as negatives, and DANR may upweight them.

  3. The gains are consistent. EvoNS is best on every reported metric on all seven benchmarks, and the ablations cover all seven datasets as well (Tables 4, 8 and 9).

Critical weaknesses
  1. I don't think the method is novel and the paper's own appendix makes that pretty clear. Eq 3 is RotatE's self-adversarial weighting applied to a margin loss. Appendix A states that RotatE itself has already evaluated that weighting "with both its negative sampling loss and a margin based ranking criterion". At τ = 0 DANR is the standard per-negative hinge as Appendix E.4 confirms. HCNS is type constrained sampling plus filtering of the negatives against the known training triples.

  2. The retention story doesn't work with the model. The historical parameters are all locked down during each update (Section 4.3, Algorithm 1, F.1) and a test set from snapshot j is ranked over EjE_j (Appendix C.3). As far as I can see, once snapshot j is complete then the score of every old test triple is fixed. EvoNS can't forget by design. It also means that a negative which is meant to reassert an earlier fact (e.g. the Daniel Radcliffe triple in Figure 1a) will consist of nothing but frozen parameters. Its distance is a constant and so it can't overwrite the stored knowledge.

What such a collision actually does is push the new entity to beat a true fact by γ\gamma. DANR then gives it the highest weight as it has the smallest distance. That is a genuine interaction between HCNS and DANR, but it's a false negative problem for learning new entities rather than a retention problem. The 6.97% to 21.51% of 10-negative groups in Figure 1b translates to roughly 0.7% to 2.4% per negative if the draws are independent. The 'historical' part isn't separated out in TtriT^tr_i as it includes the facts of the current snapshot. On FB-CKGE removing HCNS has absolutely no effect on MRR (0.231 in both rows of Table 8).

  1. A huge amount of the setup is missing. I can't find the embedding dimension, the margin γ\gamma, the low rank rank, the τ\tau and ρ values chosen for each dataset or any of the baseline hyperparameters. Section 5.1.2 refers to C.4 for the baseline settings and C.4 only covers EvoNS. A number of the gains are very small (0.007 MRR on FB-CKGE, 0.010 on GraphLower). It's not clear whether the baselines used the same 10:1 negative ratio and filtered negatives. Hits@10 is present in Tables 3 and 4 but is absent from Table 2 so "leading all 12 reported link prediction comparisons" in Table 2 is only with respect to three metrics.

  2. The efficiency claim is entirely cherry-picked. The abstract states training time is reduced by "up to 18.92%". Looking at Tables 2 and 3 though, EvoNS is actually slower than FastKGE on 4 of the 7 datasets (229s vs 165s on GraphLower for instance). Across all 7 datasets the total time is 2152s vs 2110s for FastKGE. The 6.2% average reduction cited in Section 5.2.1 is based on just the 4 datasets in Table 2. HCNS adds work per step so any speedup must come from early stopping (patience 3) firing earlier and in a single run that's all just noise. I'd completely drop efficiency as a contribution.

  3. The method is very fragile in precisely the way it's intended to be. On RELATION and HYBRID, high temperature combined with full relation mixing leads to a complete collapse of performance: RELATION MRR plummets to around 0.1 at τ=2, ρ=1 (Figure 4b) as opposed to the 0.185 reported. The relation specific pool with such sharp weighting is going to focus on the most plausible candidates and those are precisely the ones that are also the most likely to be unobserved true facts (F.2 says as much).

  4. The comparison omits the more recent CKGE methods that the paper itself references in the related work. IncDE, SAGE, ERPP, BAKE, PAAL and MF-CKGE are all mentioned in the related work and IncDE's benchmarks are used, but none of them are actually a baseline. Everything else is TransE on the FB15k-237 splits with five snapshots. EvoNS only changes the training signal so the natural thing to do would be to plug HCNS and DANR into LKGE or STARK to demonstrate that it generalises.

Questions for authors
  1. This one decides my score. What loss do the baselines and "w/o DANR" use?

  2. With frozen historical parameters and EjE_j as the candidate set, is MRRi,jMRR_{i,j} equal to MRRj,jMRR_{j,j} for every i>ji > j ? If it is, what does history consistency protect?

  3. Is the backbone FastKGE's incremental LoRA, and does removing both HCNS and DANR reproduce FastKGE's numbers?

  4. Why aren't IncDE, SAGE, ERPP or BAKE in the comparison, and why is Hits@10 missing from Table 2?

Comments on the paper

1 comment

Dear reviewers,

The authors have submitted their rebuttals. Please take a look and interact with the authors if necessary.

Thanks a lot.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.