- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies negative sampling for continual knowledge graph embedding and proposes EvoNS, which consists of two components:
- HCNS: It filters negative samples that conflict with previously observed facts using accumulated historical knowledge and relation-specific type constraints.
- DANR: It adaptively assigns greater weight to difficult negatives based on the current model’s scores and applies a separate ranking loss to each positive-negative pair.
Experiments on seven continual knowledge graph benchmarks show improvements in MRR, Hits@K, and training efficiency over existing continual learning methods.
Critical strengths
- The paper addresses a meaningful negative-sample collision problem with clear motivation.
- The method is simple and can be readily incorporated into continual knowledge graph embedding frameworks.
- The evaluation covers seven datasets and includes predictive performance, efficiency, ablation, and sensitivity analyses.
- Both HCNS and DANR provide consistent improvements, with DANR making the larger contribution.
Critical weaknesses
- The methodological novelty is limited. HCNS is essentially filtered negative sampling over accumulated historical facts combined with relation-specific type constraints. DANR is closely related to existing self-adversarial negative sampling, hard-negative weighting, and per-sample margin-ranking losses. The main contribution therefore appears to be applying and combining existing ideas in the continual knowledge graph setting, rather than introducing a fundamentally new negative learning mechanism.
- The most relevant negative-sampling baselines are missing. The experiments compare EvoNS primarily with continual learning methods, but do not directly compare it with strong negative-sampling approaches—such as RotatE-style self-adversarial sampling, KBGAN, NSCaching, and type-constrained sampling—under the same TransE/CKGE framework. Since the ablation study indicates that most of the improvement comes from DANR, these comparisons are essential for establishing both novelty and effectiveness.
- Statistical significance is not reported. On some datasets, the absolute MRR improvements are relatively small, approximately 0.007–0.011. The paper does not report results over multiple random seeds, standard deviations, or significance tests. Given the inherent randomness of negative sampling, single-run results are insufficient to demonstrate that these improvements are reliable.
- The practical contribution of HCNS appears modest. The ablation results show that removing HCNS causes a substantially smaller degradation than removing DANR. Moreover, checking whether a candidate negative conflicts with previously observed facts is a natural filtering operation. The paper should more clearly explain the substantive difference between HCNS and standard filtered sampling.
Questions for authors
- How does EvoNS compare with standard self-adversarial negative sampling under the same TransE and continual learning framework?
- What is the performance when using only cumulative filtered sampling, without relation-specific type constraints?
- Are the reported results based on multiple random runs? If so, please report the means and standard deviations.
- Can EvoNS be integrated with methods such as STARK and FastKGE, instead of being evaluated only as a separate approach?
- What are the memory costs of the historical fact index and the relation-specific lookup tables used by HCNS?
Dear reviewers,
The authors have submitted their rebuttals. Please take a look and interact with the authors if necessary.
Thanks a lot.
AC