- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes RF-LOGIC, a lightweight framework for link prediction on temporal knowledge graphs (TKGs). For each entity it keeps a fixed-size outgoing-edge memory, extracts one-hop rule features based on recency and frequency, adds symmetric shared-object rules (Eq. 4) and “negation rules” with negative weights, and scores candidates with a relation-specific linear logistic model. The stated contributions are: (1) a memory and rule-scoring scheme with fixed, scalable memory; (2) two new rule types; and (3) evaluation on the ten discrete- and continuous-time datasets in TGB 2.0, with a reported 52.2% MRR gain over CognTKE on Polecat and the ability to run at scales where neural methods run out of memory or time on Wikidata, GitHub, and Myket.
Critical strengths
- It runs on datasets that competitors cannot. On Wikidata, GitHub, and Myket, where neural methods OOM or time out, RF-LOGIC completes prediction with O(|E|k) fixed memory, and on Polecat it improves MRR by 52.2% over CognTKE. Means and standard deviations over three runs are reported. Together, these results make a useful point: a simple, reproducible baseline is still underappreciated in this area.
- The rule weights are interpretable. On the Forum dataset, the positive and negative weights line up with the ECR forum event semantics — participating in an event versus canceling participation — which is easier to audit and transfer than a black-box neural model. The appendix also provides the full hyperparameter settings and an interpretability case study.
Critical weaknesses
- The novelty is not clearly positioned and the closest prior work is missing. There is no dedicated related-work section, and CountTRuCoLa (Gastinger et al., 2025), which also uses simple rules with recency/frequency confidence and reaches state-of-the-art results on nine datasets, is not cited. The shared-object feature in Eq. (4) is essentially a common-neighbor/Jaccard feature (TNCN already used common neighbors for temporal-graph link prediction on TGB), and the “negation rule” is just a negative linear coefficient with no formal negation semantics. Calling the model “hybrid” is also inaccurate, since it has no neural component.
- Key claims are not backed by experiments. Neither of the two new rule components is ablated on its own; only the regularization coefficient λ and the loss function are ablated. Several margins are within noise — Hits@3 on Smallpedia is only 0.3 points higher and MRR on Wikidata only 1.5 points higher, without significance tests — and the model actually loses to TGNedge on Forum Hits@10.
Questions for authors
- Could you add a direct comparison with CountTRuCoLa (Gastinger et al., 2025) on the same datasets and splits? What is the essential difference from that work?
- Please add ablations that remove the shared-object feature block and the negative weights, along with a pure common-neighbor baseline, so that the contribution of each new rule type can be assessed separately.
- For the close results on Smallpedia and Wikidata, can you provide paired bootstrap significance tests? And how do you explain the loss to TGNedge on Forum Hits@10?
- What is the formal semantics of the negative-weight “negation rule”? How can a genuine negation rule be distinguished from a negative coefficient that simply fits a spurious correlation?
3 comments
Response for Reviewer cYmB Part 2
2. Shared-object features, common neighbors, and component ablations
Neighborhood overlap is an established signal. TNCN [2] learns pairwise representations from temporal common neighbors. RF-LOGIC represents overlap explicitly for each historical relation, using outgoing neighborhoods and symmetric normalization. For relation-specific neighbor sets and , let , , and :
| Feature | Formula |
|---|---|
| Jaccard | |
| RF-LOGIC shared-object feature |
Each RF-LOGIC ratio is defined as zero when its denominator is zero. The contribution is the use of these relation-conditioned features as explicit temporal rule evidence, jointly weighted with direct recency-frequency evidence for each target relation.
Shared-object removal and pure common-neighbor baseline. We remove the entire shared-object block on Forum, where the learned weights indicate strong reliance on this signal. We also report the requested pure common-neighbor baseline:
| Split | Model | MRR | Hits@1 | Hits@3 | Hits@10 |
|---|---|---|---|---|---|
| Validation | RF-LOGIC | 0.736 | 0.695 | 0.751 | 0.806 |
| Validation | RF-LOGIC, no shared-object block | 0.626 | 0.615 | 0.623 | 0.623 |
| Validation | Pure common-neighbor baseline | 0.211 | 0.148 | 0.219 | 0.298 |
| Test | RF-LOGIC | 0.739 | 0.697 | 0.753 | 0.809 |
| Test | RF-LOGIC, no shared-object block | 0.632 | 0.619 | 0.629 | 0.629 |
| Test | Pure common-neighbor baseline | 0.206 | 0.144 | 0.212 | 0.292 |
Removing shared-object features reduces test MRR by 0.107 (14.5%) and Hits@10 by 0.180 (22.2%). The full model also substantially exceeds the pure common-neighbor baseline. Together, these comparisons demonstrate the contribution of shared-object evidence within RF-LOGIC and the predictive value of the combined model on Forum. The feature-removal experiment appears in Appendix E.4, Table 15.
Non-negative weights. We constrain every relation-specific weight vector after each optimizer update:
We evaluate this restriction on GitHub, where RF-LOGIC learns strong negative diagonal weights, and observe no change in performance. Only approximately 0.5% of sampled negative destinations previously appeared in the queried subject-relation-object triple, leaving few opportunities for those diagonal weights to suppress evaluated negatives. This provides a plausible explanation for the unchanged performance. Appendix E.5 reports this ablation.
As specified at the start of Appendix E, each ablated model is trained from scratch with the same remaining hyperparameters and training pipeline. The removal or constraint is enforced throughout training and evaluation, and each checkpoint is selected independently by validation MRR.
3. Paired bootstrap tests and Forum Hits@10
We performed paired bootstrap tests of validation Hits@3 on Smallpedia against TLogic and test MRR on Wikidata against EdgeBank unlimited. Within each comparison, both methods used identical queries and TGB 2.0 negative samples. We used 10,000 bootstrap replicates, jointly resampling the two methods' per-query Hits@3 indicators on Smallpedia and reciprocal ranks on Wikidata.
| Dataset | Metric | Comparator | 95% bootstrap CI for RF-LOGIC minus baseline |
|---|---|---|---|
| Smallpedia | Hits@3 | TLogic | [−0.0016, +0.0001] |
| Wikidata | MRR | EdgeBank unlimited | [+0.0090, +0.0094] |
The Smallpedia confidence interval includes zero, indicating that the difference in Hits@3 between RF-LOGIC and TLogic is not statistically significant, despite TLogic's higher reported validation performance. In contrast, the Wikidata interval excludes zero, supporting a statistically significant improvement for RF-LOGIC over EdgeBank unlimited in test MRR.
On Forum, TGNedge-type achieves higher test Hits@10 (0.864 versus 0.809), while RF-LOGIC achieves higher Hits@1 (0.697 versus 0.229), Hits@3 (0.753 versus 0.609), and MRR (0.739 versus 0.394). These metrics describe different ranking profiles: RF-LOGIC places the correct object first much more often, while TGNedge-type places it within the top ten on more queries. RF-LOGIC's advantage is strongest at the top of the ranking, with a measurable trade-off in top-ten coverage.
Response for Reviewer cYmB Part 3
4. Meaning and interpretation of negative-weight rules
We use "negation" with statistical scoring semantics. For score , increasing a rule-body feature by , with other features fixed, changes the score by
A negative weight therefore makes the observed body evidence against the target relation. The following notation expresses this suppressive contribution for direct and shared-object bodies, respectively:
Signed weights expand the available rankings: if for all , every non-negative scorer satisfies . A negative coefficient can reverse this ordering. Appendix E.5 gives the corresponding expressivity argument.
Coefficient sign establishes the direction of the model's response. Establishing a non-spurious dependency requires independent predictive evidence. In GitHub, negative diagonal weights suppress exact-triple recurrence. Appendix D reports an event-weighted exact-triple repetition rate of 0.010 on GitHub, compared with 0.874 on Smallpedia, consistent with this interpretation. The GitHub ablation establishes that allowing suppression leaves the measured performance unchanged in this evaluation. We interpret the learned coefficients as auditable statistical evidence, with their empirical contribution assessed through the ablations above.
Thank you for the constructive suggestions, which helped us strengthen the evaluation and clarify RF-LOGIC’s contribution. We hope the additional ablations, comparisons, and statistical analyses address your concerns, and we welcome any remaining questions during the discussion. If the revised manuscript and these results resolve your main concerns, we would appreciate your reconsideration of the score.
References
[1] Gastinger, Meilicke, and Stuckenschmidt. CountTRuCoLa: Rule Learning for Interpretable Temporal Knowledge Graph Forecasting.
[2] Zhang, Wang, Wang, and Zhang. Efficient Neural Common Neighbor for Temporal Graph Link Prediction.
Response for Reviewer cYmB
Dear Reviewer cYmB,
Thank you for recognizing RF-LOGIC's scalability and interpretability. Your comments helped us sharpen the positioning and evaluate the proposed components separately. We address your four questions below.
1. Comparison with CountTRuCoLa and positioning of the contribution
We have added CountTRuCoLa [1] to the revised manuscript. The following comparison uses its published TGB 2.0 test results on the four TKG datasets and our additional run of its implementation on Software.
RF-LOGIC achieves higher values across all four metrics on Polecat, ICEWS, and Software. On Polecat, the relative MRR improvement over CountTRuCoLa is 65.6%. Further THG comparisons were limited by our available CPU RAM.
The essential differences are two predictive capabilities:
Learned evidence against the head relation. For otherwise identical descriptors, activating feature dj with wr,j<0 lowers RF-LOGIC's score by ∣wr,j∣dj. CountTRuCoLa clips rule confidences to [0,1] and combines them through decayed noisy-or. Holding existing confidences fixed, adding a firing rule cannot lower the aggregate score.
Dependence on both entities' histories. Suppose user x posted to thread z. If candidate y also posted to z, RF-LOGIC's shared-object feature is 1 when each user has one thread, if y posted to another thread, it is 0. CountTRuCoLa's temporal rules have one body atom involving the query subject. With learned parameters and the subject's history fixed, they cannot test this change in shared-thread membership.
We have also removed the "hybrid framework" characterization. Sections 1 and 3.4 now describe RF-LOGIC as a relation-specific linear scorer over predefined temporal rule templates.