- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a temporal residual memory method to mitigate catastrophic forgetting in continual knowledge graph embedding. Using BoxE as the backbone, the method stores snapshot-wise entity residuals, freezes the historical residuals while learning the current residuals, and aggregates the effective memory according to the first appearance time of each entity, which is then integrated into the current scoring process. Furthermore, a retention trajectory evaluation metric is introduced to measure the retention of old knowledge during subsequent learning.
Critical strengths
- The management strategy for historical and current residuals is clearly defined. Freezing historical residuals prevents them from being directly overwritten during current training while preserving the adaptation capacity of the current residuals. In Table 5, when freezing is disabled, the retention trajectory metric values turn negative across all seven settings, supporting the role of the freezing design in the tested configurations.
- The experiments provide controlled evidence for the memory mechanism. Same-backbone ablations show that the full method achieves higher final ranking performance than the memory-free variant across all seven settings; the extension experiments on TransE and DistMult further offer supplementary evidence under alternative scoring backbones. These results support the effectiveness of the proposed mechanism in the reported configurations.
- The manuscript is well organized, with the methodology, experiments, and limitations addressed in dedicated sections.
Critical weaknesses
-
The contribution attributable to the newly introduced mechanism remains insufficiently prominent. As shown in Tables 2 and 5, under the entity, relation, and mixed growth settings, the memory-free BoxE variant equipped with drift regularization already achieves higher MRR than ERPP. While the incorporation of the memory mechanism yields further gains, the overall ranking advantage also benefits from the backbone architecture and the regularization design. Therefore, its distinctive contribution relative to existing strategies warrants more explicit clarification.
-
The anti-forgetting gains are not consistent across settings. In Table 5, the full method attains lower RTE values than the memory-free variant under certain settings. Although the authors have already discussed this phenomenon, the trade-off between predictive performance improvement and knowledge retention warrants further clarification.
Other issues: A figure is recommended to visually illustrate how performance on previously learned knowledge evolves as learning progresses. In addition, the reference list contains duplicate entries and formatting inconsistencies that require careful revision.
Questions for authors
See Weaknesses
2 comments
Weakness 2. The anti-forgetting gains are not consistent across settings. In Table 5, the full method attains lower RTE values than the memory-free variant under certain settings. Although the authors have already discussed this phenomenon, the trade-off between predictive performance improvement and knowledge retention warrants further clarification.
With margin loss, adding memory lowered RTE on RELATION, HYBRID, and GraphHigher. RTE averages both gains and drops relative to each snapshot's initial MRR. To understand why RTE decreased, we separately computed the gains above the initial MRR and the drops below it.
We use the relative change from Eq. (1) and the test-set weight from Eq. (2). Here, is the snapshot to which the test set belongs, and is the snapshot through which the model has been trained. We define the weighted gain and weighted drop as follows.
\sum_{s=i+1}^{S} \max(\delta_i(s),0).$$ $$\mathrm{RTE}^{-} = \sum_{i=1}^{S-1} w_i \frac{1}{S-i} \sum_{s=i+1}^{S} \max(-\delta_i(s),0).$$ Gains above a snapshot's initial MRR contribute to the first component, and the magnitudes of drops below it contribute to the second. For each snapshot, we average both components over all $S-i$ subsequent evaluations and then weight them by test-set size. This gives $$\mathrm{RTE}=\mathrm{RTE}^{+}-\mathrm{RTE}^{-}.$$ | Benchmark | Weighted gain (RTE⁺) | Weighted drop (RTE⁻) | RTE | | ----------- | -------------------- | -------------------- | --------------- | | RELATION | 2.620% → 1.922% | 0.355% → 0.000% | 2.265% → 1.922% | | HYBRID | 1.758% → 0.922% | 0.902% → 0.573% | 0.855% → 0.349% | | GraphHigher | 3.009% → 2.008% | 0.000% → 0.183% | 3.009% → 1.825% | Each cell shows the change from BoxE without memory to TRM. RTE is computed from the two components before rounding. On RELATION and HYBRID, adding memory reduced the weighted drop. RTE still decreased because the weighted gain fell by more than the weighted drop did. On RELATION, TRM matched or exceeded each snapshot's initial MRR in every subsequent evaluation. On HYBRID, TRM fell below the initial MRR on the second snapshot's test set in a subsequent evaluation. It reduced the drop on the fourth snapshot, whose test set was about five times as large, so the overall weighted drop decreased. On GraphHigher, the weighted gain decreased and the weighted drop increased, with MRR falling below its initial value on the third snapshot's test set. We propose reporting RTE⁺ and RTE⁻ alongside RTE. When RTE decreases, the two components show whether gains above initial performance became smaller or drops below it became larger. With the same margin loss, adding memory changed RTE from negative to positive on FACT and GraphLower. With negative sampling loss, adding memory to the same BoxE backbone improved both RTE and ROD on all seven benchmarks. The point of Tables 11 and 12 is that residual memory can mitigate forgetting of knowledge learned in earlier snapshots even when the BoxE backbone is held fixed. > **Other issues.** A figure is recommended to visually illustrate how performance on previously learned knowledge evolves as learning progresses. In addition, the reference list contains duplicate entries and formatting inconsistencies that require careful revision. We checked the reference list and found duplicate entries for the paper on a memory-based gradient method, the broader survey of continual learning, and the survey of knowledge graph embedding from a representation-space perspective. We will consolidate the duplicate citation keys and standardize the formatting of author names, venues, titles, and page numbers.
Thank you for highlighting the separation between frozen past residuals and trainable current residuals, the evidence from the same-backbone ablations and the TransE and DistMult extensions, and the organization of the manuscript.
TRM uses separate residuals to preserve past information and adapt to new knowledge. We explain below how this design can support retention, with the ablations providing empirical evidence for its effect.
In standard sequential learning, the loss on a new snapshot directly updates existing entity representations. Learning new facts can therefore overwrite representations formed in earlier snapshots. TRM maintains a separate entity-side residual for each snapshot. During training on a new snapshot, only that snapshot's residuals are trainable among the stored residuals. Earlier residuals are frozen and retain the corrections they learned. The current residuals can thus capture changes associated with new facts while the past residuals continue to support retention.
BoxE's architecture gives these residuals a natural place in the scoring process. It represents each entity with a point and a bump, and each relation with separate regions for the head and tail roles. An entity's bump modifies the role-specific representation of its counterpart. TRM adds the snapshot-wise residuals to this bump, allowing the current cross-bump computation to reuse interaction corrections learned in earlier snapshots while retaining BoxE's relation-side scoring structure.
The scoring geometry also helps explain why this mechanism works well with BoxE. In TransE, the translated head representation is trained to approach a specific tail point. In BoxE, the representation augmented with residual memory is trained for compatibility with a relation-specific region. When the model combines corrections from different snapshots, the region gives it more room to accommodate different changes in entity interactions than alignment to a single target point would. We view this as a property of BoxE that makes it particularly suitable for snapshot-wise entity residuals.
The ablations clarify what the memory adds. The w/o Memory variant already performs well using BoxE's point-bump scoring and drift regularization. With the same backbone and regularization, full TRM achieves higher final MRR on all seven datasets and maintains positive RTE on each. In w/o Freeze, where past residuals can still be updated during later training, MRR is lower than for full TRM and RTE is negative on all seven datasets. These results support the role of freezing past residuals in retention, beyond the effect of adding residual vectors alone.
The same principle applies to TransE and DistMult. Creating residuals for each snapshot, freezing past residuals, and reusing them in the current entity representations improves ranking performance for both backbones and improves retention in most settings. The magnitude of the benefit depends on the backbone's scoring geometry.