acceptodds
Under review as a conference paper at ICLR 2027

RF-Logic: Scalable Recency-Frequency Logic Rules Learning for Interpretable Temporal Knowledge Graph Forecasting

Abstract

Temporal knowledge graph forecasting has important applications in recommendation systems, geopolitical forecasting, healthcare, and beyond. Recent works address this task by combining temporal logic rule learning with neural networks. However, existing methods often compromise interpretability when incorporating neural components and do not scale well to large temporal knowledge graphs. In this work, we introduce RF-Logic, a scalable and interpretable model for temporal knowledge graph forecasting that provides explanations through temporal logic rules. Moreover, we introduce two new types of temporal logic rules for temporal knowledge graph forecasting that RF-Logic can express: (i) symmetric shared-object rules and (ii) negation rules. In addition, we propose a dedicated training scheme for RF-Logic that improves generalization while preserving interpretability. To demonstrate the effectiveness of RF-Logic, we evaluate it on TGB 2.0 and show that it performs on par with, or exceeds the state-of-the-art methods across multiple temporal knowledge graph forecasting metrics.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
graph neural network, dynamic graphs, knowledge graphs, temporal knowledge graphs, logic rules

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cYmBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RF-LOGIC, a lightweight framework for link prediction on temporal knowledge graphs (TKGs). For each entity it keeps a fixed-size outgoing-edge memory, extracts one-hop rule features based on recency and frequency, adds symmetric shared-object rules (Eq. 4) and “negation rules” with negative weights, and scores candidates with a relation-specific linear logistic model. The stated contributions are: (1) a memory and rule-scoring scheme with fixed, scalable memory; (2) two new rule types; and (3) evaluation on the ten discrete- and continuous-time datasets in TGB 2.0, with a reported 52.2% MRR gain over CognTKE on Polecat and the ability to run at scales where neural methods run out of memory or time on Wikidata, GitHub, and Myket.

Critical strengths
  1. It runs on datasets that competitors cannot. On Wikidata, GitHub, and Myket, where neural methods OOM or time out, RF-LOGIC completes prediction with O(|E|k) fixed memory, and on Polecat it improves MRR by 52.2% over CognTKE. Means and standard deviations over three runs are reported. Together, these results make a useful point: a simple, reproducible baseline is still underappreciated in this area.
  2. The rule weights are interpretable. On the Forum dataset, the positive and negative weights line up with the ECR forum event semantics — participating in an event versus canceling participation — which is easier to audit and transfer than a black-box neural model. The appendix also provides the full hyperparameter settings and an interpretability case study.
Critical weaknesses
  1. The novelty is not clearly positioned and the closest prior work is missing. There is no dedicated related-work section, and CountTRuCoLa (Gastinger et al., 2025), which also uses simple rules with recency/frequency confidence and reaches state-of-the-art results on nine datasets, is not cited. The shared-object feature in Eq. (4) is essentially a common-neighbor/Jaccard feature (TNCN already used common neighbors for temporal-graph link prediction on TGB), and the “negation rule” is just a negative linear coefficient with no formal negation semantics. Calling the model “hybrid” is also inaccurate, since it has no neural component.
  2. Key claims are not backed by experiments. Neither of the two new rule components is ablated on its own; only the regularization coefficient λ and the loss function are ablated. Several margins are within noise — Hits@3 on Smallpedia is only 0.3 points higher and MRR on Wikidata only 1.5 points higher, without significance tests — and the model actually loses to TGNedge on Forum Hits@10.
Questions for authors
  1. Could you add a direct comparison with CountTRuCoLa (Gastinger et al., 2025) on the same datasets and splits? What is the essential difference from that work?
  2. Please add ablations that remove the shared-object feature block and the negative weights, along with a pure common-neighbor baseline, so that the contribution of each new rule type can be assessed separately.
  3. For the close results on Smallpedia and Wikidata, can you provide paired bootstrap significance tests? And how do you explain the loss to TGNedge on Forum Hits@10?
  4. What is the formal semantics of the negative-weight “negation rule”? How can a genuine negation rule be distinguished from a negative coefficient that simply fits a spurious correlation?
3 comments
AuthorsOpenReview ↗

Response for Reviewer cYmB

Dear Reviewer cYmB,

Thank you for recognizing RF-LOGIC's scalability and interpretability. Your comments helped us sharpen the positioning and evaluate the proposed components separately. We address your four questions below.

1. Comparison with CountTRuCoLa and positioning of the contribution

We have added CountTRuCoLa [1] to the revised manuscript. The following comparison uses its published TGB 2.0 test results on the four TKG datasets and our additional run of its implementation on Software.

DatasetModelTest MRRTest Hits@1Test Hits@3Test Hits@10
SmallpediaCountTRuCoLa0.6440.5950.6870.717
SmallpediaRF-LOGIC0.6350.5850.6830.707
PolecatCountTRuCoLa0.2560.1780.2910.408
PolecatRF-LOGIC0.4240.3190.4910.624
ICEWSCountTRuCoLa0.2140.1570.2410.321
ICEWSRF-LOGIC0.2620.1630.2980.458
WikidataCountTRuCoLa0.6090.5980.6160.628
WikidataRF-LOGIC0.5500.5350.5600.562
SoftwareCountTRuCoLa0.4350.4280.4400.442
SoftwareRF-LOGIC0.4980.4960.4980.498

RF-LOGIC achieves higher values across all four metrics on Polecat, ICEWS, and Software. On Polecat, the relative MRR improvement over CountTRuCoLa is 65.6%. Further THG comparisons were limited by our available CPU RAM.

The essential differences are two predictive capabilities:

Learned evidence against the head relation. For otherwise identical descriptors, activating feature djd_j with wr,j<0w_{r,j}<0 lowers RF-LOGIC's score by ∣wr,j∣dj|w_{r,j}|d_j. CountTRuCoLa clips rule confidences to [0,1][0,1] and combines them through decayed noisy-or. Holding existing confidences fixed, adding a firing rule cannot lower the aggregate score.

Dependence on both entities' histories. Suppose user xx posted to thread zz. If candidate yy also posted to zz, RF-LOGIC's shared-object feature is 1 when each user has one thread, if yy posted to another thread, it is 0. CountTRuCoLa's temporal rules have one body atom involving the query subject. With learned parameters and the subject's history fixed, they cannot test this change in shared-thread membership.

We have also removed the "hybrid framework" characterization. Sections 1 and 3.4 now describe RF-LOGIC as a relation-specific linear scorer over predefined temporal rule templates.

AuthorsOpenReview ↗

Response for Reviewer cYmB Part 2

2. Shared-object features, common neighbors, and component ablations

Neighborhood overlap is an established signal. TNCN [2] learns pairwise representations from temporal common neighbors. RF-LOGIC represents overlap explicitly for each historical relation, using outgoing neighborhoods and symmetric normalization. For relation-specific neighbor sets AA and BB, let a=∣A∣a=|A|, b=∣B∣b=|B|, and c=∣A∩B∣c=|A\cap B|:

FeatureFormula
Jaccardc/(a+b−c)\displaystyle c/(a+b-c)
RF-LOGIC shared-object feature12(c/a+c/b)\displaystyle \frac12\left(c/a+c/b\right)

Each RF-LOGIC ratio is defined as zero when its denominator is zero. The contribution is the use of these relation-conditioned features as explicit temporal rule evidence, jointly weighted with direct recency-frequency evidence for each target relation.

Shared-object removal and pure common-neighbor baseline. We remove the entire shared-object block ds,o(2)d^{(2)}_{s,o} on Forum, where the learned weights indicate strong reliance on this signal. We also report the requested pure common-neighbor baseline:

SplitModelMRRHits@1Hits@3Hits@10
ValidationRF-LOGIC0.7360.6950.7510.806
ValidationRF-LOGIC, no shared-object block0.6260.6150.6230.623
ValidationPure common-neighbor baseline0.2110.1480.2190.298
TestRF-LOGIC0.7390.6970.7530.809
TestRF-LOGIC, no shared-object block0.6320.6190.6290.629
TestPure common-neighbor baseline0.2060.1440.2120.292

Removing shared-object features reduces test MRR by 0.107 (14.5%) and Hits@10 by 0.180 (22.2%). The full model also substantially exceeds the pure common-neighbor baseline. Together, these comparisons demonstrate the contribution of shared-object evidence within RF-LOGIC and the predictive value of the combined model on Forum. The feature-removal experiment appears in Appendix E.4, Table 15.

Non-negative weights. We constrain every relation-specific weight vector after each optimizer update:

wr←max⁡(wr,0).w_r\leftarrow\max(w_r,0).

We evaluate this restriction on GitHub, where RF-LOGIC learns strong negative diagonal weights, and observe no change in performance. Only approximately 0.5% of sampled negative destinations previously appeared in the queried subject-relation-object triple, leaving few opportunities for those diagonal weights to suppress evaluated negatives. This provides a plausible explanation for the unchanged performance. Appendix E.5 reports this ablation.

As specified at the start of Appendix E, each ablated model is trained from scratch with the same remaining hyperparameters and training pipeline. The removal or constraint is enforced throughout training and evaluation, and each checkpoint is selected independently by validation MRR.

3. Paired bootstrap tests and Forum Hits@10

We performed paired bootstrap tests of validation Hits@3 on Smallpedia against TLogic and test MRR on Wikidata against EdgeBank unlimited. Within each comparison, both methods used identical queries and TGB 2.0 negative samples. We used 10,000 bootstrap replicates, jointly resampling the two methods' per-query Hits@3 indicators on Smallpedia and reciprocal ranks on Wikidata.

DatasetMetricComparator95% bootstrap CI for RF-LOGIC minus baseline
SmallpediaHits@3TLogic[−0.0016, +0.0001]
WikidataMRREdgeBank unlimited[+0.0090, +0.0094]

The Smallpedia confidence interval includes zero, indicating that the difference in Hits@3 between RF-LOGIC and TLogic is not statistically significant, despite TLogic's higher reported validation performance. In contrast, the Wikidata interval excludes zero, supporting a statistically significant improvement for RF-LOGIC over EdgeBank unlimited in test MRR.

On Forum, TGNedge-type achieves higher test Hits@10 (0.864 versus 0.809), while RF-LOGIC achieves higher Hits@1 (0.697 versus 0.229), Hits@3 (0.753 versus 0.609), and MRR (0.739 versus 0.394). These metrics describe different ranking profiles: RF-LOGIC places the correct object first much more often, while TGNedge-type places it within the top ten on more queries. RF-LOGIC's advantage is strongest at the top of the ranking, with a measurable trade-off in top-ten coverage.

AuthorsOpenReview ↗

Response for Reviewer cYmB Part 3

4. Meaning and interpretation of negative-weight rules

We use "negation" with statistical scoring semantics. For score ϕr(d)=wr⊤d\phi_r(d)=w_r^\top d, increasing a rule-body feature by δ>0\delta>0, with other features fixed, changes the score by

ϕr(d+δej)−ϕr(d)=wr,jδ.\phi_r(d+\delta e_j)-\phi_r(d)=w_{r,j}\delta.

A negative weight therefore makes the observed body evidence against the target relation. The following notation expresses this suppressive contribution for direct and shared-object bodies, respectively:

¬r(s,o,t+)←r′(s,o,t),t<t+,\neg r(s,o,t^+)\leftarrow r'(s,o,t),\qquad t<t^+,

¬r(s,o,t+)←r′(s,x,t1)∧r′(o,x,t2),t1,t2<t+.\neg r(s,o,t^+)\leftarrow r'(s,x,t_1)\land r'(o,x,t_2),\qquad t_1,t_2<t^+.

Signed weights expand the available rankings: if uj≥vju_j\ge v_j for all jj, every non-negative scorer satisfies wr⊤u≥wr⊤vw_r^\top u\ge w_r^\top v. A negative coefficient can reverse this ordering. Appendix E.5 gives the corresponding expressivity argument.

Coefficient sign establishes the direction of the model's response. Establishing a non-spurious dependency requires independent predictive evidence. In GitHub, negative diagonal weights suppress exact-triple recurrence. Appendix D reports an event-weighted exact-triple repetition rate of 0.010 on GitHub, compared with 0.874 on Smallpedia, consistent with this interpretation. The GitHub ablation establishes that allowing suppression leaves the measured performance unchanged in this evaluation. We interpret the learned coefficients as auditable statistical evidence, with their empirical contribution assessed through the ablations above.

Thank you for the constructive suggestions, which helped us strengthen the evaluation and clarify RF-LOGIC’s contribution. We hope the additional ablations, comparisons, and statistical analyses address your concerns, and we welcome any remaining questions during the discussion. If the revised manuscript and these results resolve your main concerns, we would appreciate your reconsideration of the score.

References

[1] Gastinger, Meilicke, and Stuckenschmidt. CountTRuCoLa: Rule Learning for Interpretable Temporal Knowledge Graph Forecasting.

[2] Zhang, Wang, Wang, and Zhang. Efficient Neural Common Neighbor for Temporal Graph Link Prediction.

Reviewer_4TCAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RF-Logic, a lightweight and interpretable framework for temporal knowledge graph forecasting. Each entity maintains a fixed-length memory of recent outgoing interactions. For each candidate, the model extracts relation-specific recency-frequency features and symmetric shared-object features, then combines them using a relation-specific linear scorer. Positive and negative weights respectively support and suppress candidate predictions, allowing the scoring process to be interpreted through temporal rule templates. Experiments cover eight temporal knowledge graph and heterogeneous graph datasets from TGB 2.0. The paper also presents weight visualizations, analyses of dataset behavior, and ablations of the recency-decay parameter and training objective.

Critical strengths
  • The model maps explicit historical features to predictions through relation-specific linear weights. This makes it possible to inspect which historical relations support or suppress a candidate, rather than relying exclusively on opaque entity representations. The weight visualizations and accompanying dataset analyses provide useful examples of this interpretability.
  • The evaluation spans several datasets with different domains and scales, and the model obtains competitive results on several of them. Its fixed-length entity memory also allows it to run on large datasets where some evaluated baselines encounter memory or time limits. Together, these results support the practical value of the lightweight design, although more detailed resource measurements would make the scalability evidence stronger.
Critical weaknesses
  1. The Forum analysis and shared-object visualizations provide meaningful evidence that these features capture relevant patterns. However, the paper does not report a controlled comparison with a model trained without the shared-object block. Consequently, it remains unclear how much predictive improvement this core component contributes beyond the direct recency-frequency features, and whether that contribution is consistent across datasets. Separate ablation studies are recommended to remove the shared-object features and constrain the scoring weights to be nonnegative, while keeping the remaining experimental settings comparable. These comparisons would help clarify the individual contributions of the two core rule components to predictive performance, beyond what can be inferred from the weight visualizations alone.
  2. Negative coefficients are central to the proposed interpretation of negation rules. The visualizations show that such coefficients are learned and can align with dataset behavior, but this does not establish their incremental predictive value. A comparison with a model trained under nonnegative weight constraints would help determine whether signed evidence materially improves forecasting over a model that only accumulates positive support. The existing decay and loss-function ablations do not resolve this question.
  3. The storage-complexity analysis and successful runs on large datasets support the feasibility of the design. Nevertheless, the paper does not provide a systematic comparison of measured runtime and peak memory against representative baselines. This would help establish the practical efficiency advantage beyond the storage-complexity analysis and the reported OOM/OOT outcomes. Persistent memory complexity alone does not capture temporary feature-computation tensors or the cost of scoring candidates. The practical size of the efficiency advantage therefore remains difficult to assess.
  4. Appendix B describes selecting a memory size that fits the resource budget and remains relevant under recency decay, but does not show how predictive performance and resource use vary with k. This matters because the direct features use recency decay, whereas the shared-object features use distinct objects within the stored history without additional decay. Memory truncation can therefore affect the two components differently, leaving the robustness of the chosen memory budget unclear. In particular, it would be useful to clarify whether shared-object features remain effective with smaller memories, since they do not use the additional recency decay applied to direct features.
  5. The evaluation datasets used in this paper differ from those commonly adopted in prior temporal knowledge graph (TKG) forecasting studies, such as ICEWS14, ICEWS18, ICEWS0515, WIKI, GDELT, and YAGO. Meanwhile, several baselines included in the comparison, such as TLogic, CognTKE, and TransFIR, have also been evaluated on commonly used benchmarks, including ICEWS14 and ICEWS18. The reasons for these differences in dataset selection are not clearly explained. In addition, relevant rule-based TKG forecasting methods, such as TempValid, and other TKG forecasting methods, such as AnRe, are missing from the baseline comparison. These limitations make it more difficult to assess the proposed method's effectiveness relative to existing approaches and its generalizability across different datasets and evaluation settings.
Questions for authors

Please refer to the Weaknesses.

5 comments
AuthorsOpenReview ↗

Response for Reviewer 4TCA

Dear Reviewer 4TCA,

Thank you for recognizing RF-Logic's interpretability and the practical value of its lightweight design. Your comments prompted controlled component ablations, measured resource comparisons, a memory-size study, and additional baseline and benchmark evaluations. We address your concerns below.

1. Contribution of the shared-object rule

We added the requested ablation in Appendix E.4, Table 15, removing the complete shared-object descriptor ds,o(2)d^{(2)}_{s,o}. We evaluate this ablation on Forum, where the learned weights and dataset analysis indicate the strongest reliance on shared-object evidence.

As specified at the beginning of Appendix E, each ablated model was trained from scratch using the same training pipeline and remaining hyperparameters as the full model. The feature removal or weight constraint was enforced throughout training and evaluation, and each model's checkpoint was selected independently using validation MRR.

ModelSplitMRRHits@1Hits@3Hits@10
RF-LogicValidation0.736 ± 0.0010.695 ± 0.0010.751 ± 0.0010.806 ± 0.001
RF-Logic without shared-object featuresValidation0.626 ± 0.0010.615 ± 0.0010.623 ± 0.0010.623 ± 0.001
RF-LogicTest0.739 ± 0.0010.697 ± 0.0010.753 ± 0.0010.809 ± 0.001
RF-Logic without shared-object featuresTest0.632 ± 0.0010.619 ± 0.0010.629 ± 0.0010.629 ± 0.001

Removing the shared-object block decreases test MRR by 0.107, from 0.739 to 0.632, and test Hits@10 by 0.180, from 0.809 to 0.629. These correspond to relative reductions of 14.5% and 22.2%, respectively. With the complete direct recency-frequency block retained and retrained, this comparison establishes a substantial predictive contribution from shared-object evidence on Forum.

2. Contribution of negative weights

We added the requested nonnegative-weight ablation in Appendix E.5. After each optimizer update, we project every relation-specific weight vector onto the nonnegative orthant:

wr←max⁡(wr,0).w_r \leftarrow \max(w_r,0).

We evaluate this variant on GitHub, where the unconstrained model learns particularly strong negative diagonal weights. MRR and Hits@1/@3/@10 remain unchanged.

A possible explanation is the limited activation of the negative diagonal direct-recurrence weights: only approximately 0.5% of sampled negative destinations previously occurred in an exact triple with the queried subject and relation. These weights therefore have few opportunities to suppress evaluated negative candidates.

Signed weights also expand the class of linear ranking functions over the descriptor space. If xj≥yjx_j\geq y_j for every coordinate and wr≥0w_r\geq0, then necessarily

wr⊤x≥wr⊤y.w_r^\top x\geq w_r^\top y.

Signed weights can reverse this ordering: for x=(1,1)x=(1,1), y=(0.5,0)y=(0.5,0), and wr=(0,−1)w_r=(0,-1),

wr⊤y=0>−1=wr⊤x.w_r^\top y=0>-1=w_r^\top x.

This establishes the additional ability to suppress candidates through negative historical evidence.

3. Measured runtime and memory

Appendix F, Table 16, now compares RF-Logic with CEN and RE-GCN, two GPU-based baselines that complete evaluation on Smallpedia, Polecat, and ICEWS. The experimental setup uses a single NVIDIA RTX A6000 GPU and an AMD Ryzen 9 7900X processor. The table reports training and test throughput in queries per second and peak GPU memory.

DatasetModelTrain q/s ↑Test q/s ↑Peak GPU memory (GB) ↓
SmallpediaCEN22,67456820.8
SmallpediaRE-GCN38,8001,2389.6
SmallpediaRF-Logic1,1395203.8
PolecatCEN3,21118522.9
PolecatRE-GCN6,23343511.0
PolecatRF-Logic1,2327050.6
ICEWSCEN7,34233020.0
ICEWSRE-GCN12,8177679.5
ICEWSRF-Logic9303139.7

RF-Logic uses substantially less training GPU memory on Smallpedia and Polecat. On ICEWS, its usage is comparable to RE-GCN and below CEN. On Polecat, RF-Logic also achieves the highest inference throughput, at 705 queries/s, or 1.62 times that of RE-GCN. Both baselines train faster on all three datasets, and their inference throughput is higher on Smallpedia and ICEWS. These measurements establish the dataset-dependent resource trade-offs alongside RF-Logic's higher test MRR and Hits@1/@3/@10 than these two baselines on all three datasets.

AuthorsOpenReview ↗

Response for Reviewer 4TCA Part 2

4. Sensitivity to memory size

Appendix E.3, Table 14, now reports performance for k∈{10,50,250,1000}k\in\{10,50,250,1000\} on Smallpedia, Polecat, Software, and Wikidata.

The two feature blocks respond differently to truncation. For direct recency-frequency features, when λ≥1\lambda\geq1,

2−λi≥∑j=i+1∞2−λj.2^{-\lambda i}\geq\sum_{j=i+1}^{\infty}2^{-\lambda j}.

Thus, an occurrence at relation-local rank ii receives at least as much unnormalized weight as all less-recent positions combined. The shared-object features use distinct objects within the retained history, with no additional decay. Changing kk can therefore change both their shared-object counts and their normalization. Our sweep includes datasets favoring strong recency, Smallpedia and Polecat, and weaker recency, Software and Wikidata.

DatasetkVal. MRRVal. Hits@1Val. Hits@3Val. Hits@10Test MRRTest Hits@1Test Hits@3Test Hits@10
Wikidata100.551 ± 0.0010.545 ± 0.0010.555 ± 0.0010.555 ± 0.0010.466 ± 0.0010.459 ± 0.0010.469 ± 0.0010.469 ± 0.001
Wikidata500.627 ± 0.0010.615 ± 0.0010.636 ± 0.0010.637 ± 0.0010.526 ± 0.0010.515 ± 0.0010.534 ± 0.0010.535 ± 0.001
Wikidata2500.646 ± 0.0010.631 ± 0.0010.656 ± 0.0010.658 ± 0.0010.550 ± 0.0010.535 ± 0.0010.560 ± 0.0010.562 ± 0.001
Wikidata10000.646 ± 0.0010.631 ± 0.0010.656 ± 0.0010.658 ± 0.0010.550 ± 0.0010.535 ± 0.0010.560 ± 0.0010.562 ± 0.001
Smallpedia100.620 ± 0.0010.594 ± 0.0010.648 ± 0.0010.652 ± 0.0010.601 ± 0.0010.563 ± 0.0010.639 ± 0.0010.648 ± 0.001
Smallpedia500.654 ± 0.0010.618 ± 0.0010.690 ± 0.0010.701 ± 0.0010.630 ± 0.0010.582 ± 0.0010.677 ± 0.0010.697 ± 0.001
Smallpedia2500.658 ± 0.0010.620 ± 0.0010.695 ± 0.0010.708 ± 0.0010.635 ± 0.0010.585 ± 0.0010.683 ± 0.0010.707 ± 0.001
Smallpedia10000.657 ± 0.0010.619 ± 0.0010.695 ± 0.0010.708 ± 0.0010.634 ± 0.0010.584 ± 0.0010.683 ± 0.0010.707 ± 0.001
Polecat100.445 ± 0.0010.354 ± 0.0010.532 ± 0.0010.584 ± 0.0010.437 ± 0.0010.343 ± 0.0010.525 ± 0.0010.581 ± 0.001
Polecat500.435 ± 0.0010.333 ± 0.0010.508 ± 0.0010.627 ± 0.0010.424 ± 0.0010.320 ± 0.0010.497 ± 0.0010.626 ± 0.001
Polecat2500.436 ± 0.0010.334 ± 0.0010.504 ± 0.0010.627 ± 0.0010.425 ± 0.0010.321 ± 0.0010.491 ± 0.0010.623 ± 0.001
Polecat10000.433 ± 0.0010.329 ± 0.0010.502 ± 0.0010.628 ± 0.0010.423 ± 0.0010.318 ± 0.0010.490 ± 0.0010.624 ± 0.001
Software100.425 ± 0.0010.424 ± 0.0010.425 ± 0.0010.425 ± 0.0010.486 ± 0.0010.485 ± 0.0010.486 ± 0.0010.486 ± 0.001
Software500.431 ± 0.0010.429 ± 0.0010.430 ± 0.0010.430 ± 0.0010.493 ± 0.0010.491 ± 0.0010.493 ± 0.0010.493 ± 0.001
Software2500.435 ± 0.0010.433 ± 0.0010.435 ± 0.0010.435 ± 0.0010.498 ± 0.0010.498 ± 0.0010.498 ± 0.0010.498 ± 0.001
Software10000.435 ± 0.0010.433 ± 0.0010.435 ± 0.0010.435 ± 0.0010.498 ± 0.0010.498 ± 0.0010.498 ± 0.0010.498 ± 0.001

Increasing kk from 50 to 250 improves Wikidata test MRR from 0.526 to 0.550, with smaller changes on the other datasets. From k=250k=250 to k=1000k=1000, the maximum absolute change across all reported metrics is 0.005, while entity-history storage increases fourfold because it scales as O(∣E∣k)O(|\mathcal E|k). Small memories already retain substantial predictive performance: at k=10k=10, test MRR is 0.601 on Smallpedia, 0.437 on Polecat, and 0.486 on Software. These results characterize the complete model's sensitivity to history truncation. Appendix B also clarifies that kk is selected using validation performance within the available memory budget.

AuthorsOpenReview ↗

Response for Reviewer 4TCA Part 3

5. Benchmark selection and additional baselines

We chose TGB 2.0 [1] for its standardized chronological evaluation, candidate sets, filtering, and tie handling, together with its coverage of large temporal graphs. Its Smallpedia and Wikidata datasets share their source with WIKI, and its ICEWS dataset shares its source with ICEWS14/18. Their extractions and coverage differ substantially: TGB ICEWS spans 1995–2022, while ICEWS14 covers 2014 and ICEWS18 covers January–October 2018. The following statistics illustrate the differences.

Wikidata-based datasetEntitiesRelationsQuadruples
WIKI12,55424669,934
TGB Smallpedia47,433283550,376
TGB Wikidata1,226,4405969,856,203
ICEWS-based datasetEntitiesRelationsQuadruples
ICEWS14 (7,128-entity version)7,12823090,730
ICEWS1823,033256468,558
TGB ICEWS87,85639115,513,446

To connect our evaluation directly to benchmarks used in prior TKG forecasting studies, we additionally evaluated RF-Logic on YAGO and GDELT.

YAGO

ModelTest MRRTest Hits@1Test Hits@3Test Hits@10
CEN0.8230.7830.8500.893
RE-GCN0.8040.7620.8310.881
EdgeBank (time window, 0.15)0.6170.4410.6850.904
EdgeBank (unlimited)0.4780.1290.6450.889
RF-Logic0.8910.8570.9270.931

GDELT

ModelTest MRRTest Hits@1Test Hits@3Test Hits@10
CEN0.16910.10730.17760.2864
RE-GCN0.16770.10680.17390.2836
EdgeBank (time window)0.01900.00170.00860.0347
EdgeBank (unlimited)0.00700.00040.00180.0085
RF-Logic0.19710.12380.21370.3442

RF-Logic achieves the highest test MRR and Hits@1/@3/@10 among these compared methods on both datasets.

We also implemented TempValid [2] and AnRe [3] from their published descriptions. The results for our TempValid implementation are below.

DatasetModelTest MRRTest Hits@1Test Hits@3Test Hits@10
SmallpediaTempValid0.5900.4580.6800.718
SmallpediaRF-Logic0.6350.5850.6830.707
YAGOTempValid0.8900.8210.9290.930
YAGORF-Logic0.8910.8570.9270.931
SoftwareTempValid0.4870.4810.4900.494
SoftwareRF-Logic0.4980.4960.4980.498

RF-Logic obtains higher reported values in 10 of the 12 dataset–metric combinations. The largest MRR gain is on Smallpedia, from 0.590 to 0.635.

AnRe [3] uses multiple LLM calls, including InternLM2-7B, to construct analogical reasoning examples and predict future objects. Its published evaluation samples 200 test events per dataset and averages three experiments (§4.1). Table 6 reports 63.82 seconds per iteration with one analogical example. Hence evaluating AnRe on the smallest TGB 2.0 dataset (100k queries) will not finish in a reasonable time.

Thank you for the suggestions that led to these additions. We hope the controlled ablations and expanded comparisons help clarify the contribution and practical scope of RF-Logic. Should you have any remaining concerns or questions, we would be grateful for the opportunity to address them in this discussion. In case that we managed to address your concerns we kindly ask that you would consider revisiting your score.

References

[1] Gastinger et al. TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs. NeurIPS 2024.

[2] Huang et al. Confidence is not Timeless: Modeling Temporal Validity for Rule-based Temporal Knowledge Graph Forecasting. ACL 2024.

[3] Tang et al. AnRe: Analogical Replay for Temporal Knowledge Graph Forecasting. ACL 2025.

Reviewer_d3W4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RF-LOGIC for entity forecasting on temporal knowledge graphs (TKG/THG): each entity keeps a fixed-size memory of its k most recent outgoing interactions; for a query (s, r, ?, t+) and candidate object o, two hand-crafted feature blocks are computed — relation-local recency-frequency features (Eq. 3) and symmetric shared-object overlap features (Eq. 5) — which a relation-specific linear scorer (Eq. 2) combines into a ranking score. Training uses a pairwise sigmoid ranking loss with ElasticNet regularization and a structured initialization. The two feature families are presented as two new types of temporal logic rules (symmetric shared-object rules; negation rules realized as negative weights); a GPU implementation (Appendix B) achieves O(|E|k) memory. On all eight TGB 2.0 datasets under the standardized protocol, RF-LOGIC wins most test cells (+52.2% over CognTKE on Polecat; +10.9% over EdgeBank on Software) and is the only learned method that scales to the largest datasets (Wikidata, Github, Myket). Weight visualizations and three case studies support interpretability claims.

Critical strengths
  1. Genuine scalability, demonstrated where it matters, with the strongest evaluation practice in its subfield: the O(|E|k) memory design runs on TGB 2.0's largest datasets (9.9M-53.6M quadruples) where every learned baseline goes OOM/OOT, under a public standardized protocol with chronological splits, shared negatives, leaderboard-anchored baselines, and three runs with std.

  2. Strong test-split results with large margins where structure exists (Polecat test MRR 0.424 vs. 0.277; Software 0.498 vs. 0.449; ICEWS test 0.262 vs. 0.211), plus interpretability analyses tied to measurable dataset statistics (e.g., negative diagonal weights matched to GitHub's measured 0.010 repetition rate) rather than heatmap-only treatments.

Critical weaknesses
  1. The framing far exceeds the mechanism, the claimed novel components are never isolated, and the categorical superiority claim is contradicted by the paper's own tables. The model is a linear scorer over two hand-crafted recency/frequency statistics — no rule mining, no confidence estimation, no neural component — yet it is presented as "temporal logic rules learning" in a "hybrid framework"; Appendix E never ablates the feature blocks or tests a non-negative-weight variant. Meanwhile "outperforms all baselines" fails on every ICEWS validation metric, Smallpedia test Hits@10 (0.707 vs. RecB_train 0.716), Wikidata Hits@3/@10 (both splits, to the memoryless EdgeBank), and Forum validation MRR (0.736 vs. 0.767) — and the paper's own boldface marks these losses, while the "+52.2%" headline silently uses the validation pair (test is +53.1%).

  2. The THG comparison rests on an undocumented baseline whose key cells are corrupted, and the scalability question is answered without measurements. TGN_edge-type is never defined yet beats RF-LOGIC on Forum validation MRR; its Forum Hits cells (and TGN's) are identical across Tables 4/8/9 and violate the arithmetic bound MRR <= (1+Hits@1)/2 against Table 3, so at least one table is wrong on the most decision-relevant comparison. No training/inference time or memory is reported anywhere, k is "the largest that fit within 40 GB", final per-dataset configurations are unreported, and on the largest THG dataset (Myket) the margins over EdgeBank are within the uniform +/-0.001 std.

Questions for authors

Clarification questions (answers could change my assessment):

  1. Please provide the win/tie/loss count of RF-LOGIC against the best baseline per cell, split by validation and test, across Tables 1-4 and 6-9.

  2. What are the per-dataset values of alpha, k, m, (lambda, mu), and the number of negatives, and how was alpha selected for the six datasets not covered by Appendix E?

  3. Please report wall-clock training/inference times and peak memory per dataset for RF-LOGIC and the runnable baselines. How does performance degrade with halved k on the largest datasets?

  4. What exactly is TGN_edge-type (source, architectural deltas, tuning)? Is it the TGB 2.0 baseline-suite variant of TGN with edge-type information?

  5. Which values are the correct Forum Hits@k for TGN and TGN_edge-type? The printed values are identical for both architectures in Tables 4, 8, 9 and arithmetically incompatible with Table 3's Forum MRRs (MRR <= (1+Hits@1)/2 is violated). Which table is wrong?

  6. How do the two "new rule types" relate to prior art: common-neighbor/co-occurrence heuristics in link prediction, and negated-body rules in non-monotonic KG rule learning (e.g., Tran et al., ILP 2016)? Has any prior temporal rule learner supported either construct?

Additional concerns (for the record):

  1. On THG datasets, how many negatives q per query does the TGB 2.0 protocol sample, and is q shared across methods? Are candidate objects restricted by node type, and is inverse-relation augmentation applied there as on TKG datasets?

  2. Why does RF-LOGIC trail the best baseline on every ICEWS validation metric while leading decisively on test? What property of the validation window or checkpoint selection explains this inversion?

  3. On Software, Github, and (nearly) Myket, Hits@1 = Hits@3 = Hits@10 — what property of the evaluation produces this profile, and how should it affect interpretation?

  4. The AI-use statement says generative AI was used to "implement methods" — which components were AI-generated, and how were they validated?

  5. Are the +/-0.001 stds exact? Is the pipeline deterministic? What explains identical-to-three-decimals Hits@3/@10 for RF-LOGIC and EdgeBank on Myket?

  6. Why is RF-LOGIC bolded over TLogic's higher value at Smallpedia validation Hits@3 (0.695 vs. 0.696)?

6 comments
AuthorsOpenReview ↗

Response to Reviewer d3W4

Dear Reviewer d3W4,

Thank you for the detailed review and for recognizing RF-LOGIC’s scalability, strong test results, and interpretable predictions. Your comments identified important gaps in framing, component isolation, and reporting. We address these below, distinguishing the evidence supported by our experiments from claims that require qualification.

1. Framing of the proposed model. We agree that the original framing did not sufficiently distinguish learning rule weights from discovering rule structures. We have revised the Introduction, contributions, and Section 3.4 to describe RF-LOGIC as a relation-specific linear scorer over two predefined feature families: direct recency–frequency evidence and symmetric shared-object evidence. Section 3.4 explicitly defines the scope of rule learning: the feature templates are fixed, and their weights are learned jointly through the forecasting objective. RF-LOGIC has no neural encoder or learned entity embeddings. Its contribution is the combination of interpretable scoring, these complementary historical features, and bounded-memory computation at the scale evaluated here.

2. Performance claims. Thank you for identifying the overstatement. RF-LOGIC does not lead on every metric or split, including ICEWS validation, Smallpedia test Hits@10, and Wikidata Hits@3/@10. Sections 4.2–4.3 now state that RF-LOGIC achieves the highest reported mean test MRR among the compared methods on all eight datasets. This does not establish superiority on every metric or statistical significance for every margin. We have also corrected the Polecat headline to the relative test MRR improvement over CognTKE: 0.424 versus 0.277, approximately 53.1%.

3. TGN and TGN-edge-type. Section 4.1 now identifies and credits the TGB 2.0 variant: it embeds edge types and concatenates these embeddings with TGN’s original messages [3, Appendix G.2]. Our original Forum tables combined published MRR values with Hits@k from our own reruns, which did not reproduce the published MRRs. The revised tables report our rerun MRRs alongside the rerun Hits metrics. The numerical values are given under Q5.

4. Memory selection and configurations. We agree that “the largest that fit within 40 GB…” was insufficient. Appendix B now explains that memory selection should consider validation performance and the memory budget. The decay profile 2−λi2^{-\lambda i} guides the retention of direct-interaction evidence, but cannot determine memory size alone: shared-object features are not temporally decayed, and direct features have uniform weights when λ=0\lambda=0. Table 6 reports the dataset-specific decay, memory limits, and training-negative counts. Q2 summarizes these settings and identifies the remaining regularization-reporting detail.

5. Margins over EdgeBank. EdgeBank is deterministic for fixed histories and evaluation candidates, but RF-LOGIC’s training varies across runs. We therefore interpret MyKet’s test MRR of 0.458 versus 0.456 as a small numerical improvement, without claiming a substantial or statistically established advantage.

AuthorsOpenReview ↗

Response to Reviewer d3W4 Part 2

6. Component ablations and memory sensitivity. Appendix E now includes removal of the shared-object block, a non-negative-weight restriction, and memory-size sensitivity. Each ablated model was trained from scratch with the same training pipeline and remaining hyperparameters, with the ablation enforced throughout training and evaluation. Checkpoints were selected independently by validation MRR.

(a) Shared-object descriptor — Appendix E.4. We remove the entire shared-object block while retaining the direct recency–frequency features. On Forum:

VariantValidation MRRTest MRRValidation Hits@10Test Hits@10
RF-LOGIC0.7360.7390.8060.809
Without shared-object features0.6260.6320.6230.629

Removing this block reduces test MRR by 0.107 (14.5% relative) and test Hits@10 by 0.180 (22.2% relative). This isolates a substantial contribution on Forum.

(b) Non-negative weights — Appendix E.5. We project every relation-specific weight vector onto the non-negative orthant after each optimizer update, wr←max⁡(wr,0)w_r\leftarrow\max(w_r,0). On GitHub, where the unconstrained model learns strong negative diagonal weights, we observed no performance difference. A possible explanation is that only approximately 0.5% of sampled evaluation negatives previously occurred as an exact triple with the queried subject and relation, limiting activation of those diagonal features.

Nevertheless, negative weights enlarge the class of ranking functions that RF-LOGIC can represent, as shown below. Proof. Allowing negative weights strictly expands the class of linear ranking functions over the descriptor space. Let xx and yy be the descriptors of two candidate destinations. If xj≥yjx_j \ge y_j for every coordinate and wr≥0w_r \ge 0, then necessarily

wr⊤x≥wr⊤y.w_r^\top x \ge w_r^\top y.

Consequently, a non-negative scorer cannot rank yy above xx, regardless of the training procedure. Signed weights can reverse this ordering: for

x=(1,1),y=(0.5,0),wr=(0,−1),x=(1,1), \qquad y=(0.5,0), \qquad w_r=(0,-1),

we obtain

wr⊤y=0>−1=wr⊤x.w_r^\top y=0>-1=w_r^\top x.

Thus, negative weights permit suppressing historical evidence and expressing candidate orderings that a non-negative scorer cannot represent.

(c) Memory-size sensitivity — Appendix E.3. We evaluate k{10,50,250,1000} on four datasets. Test MRR is:

Datasetk=10k=50k=250k=1000
Smallpedia0.6010.6300.6350.634
Polecat0.4370.4240.4250.423
Software0.4860.4930.4980.498
Wikidata0.4660.5260.5500.550

The effects are dataset dependent: Wikidata benefits from larger memory, whereas Polecat achieves its highest MRR at k=10. Increasing k from 250 to 1000 changes test MRR by at most 0.002 and any reported validation/test metric by at most 0.005, while persistent history storage grows as O(∣E∣k)O(|\mathcal E|k). These results establish sensitivity on the tested datasets; they do not directly measure halving the selected memory on GitHub or MyKet.

AuthorsOpenReview ↗

Response to Reviewer d3W4 Part 3

Responses to the clarification questions

Q1. Win/tie/loss counts. We compare RF-LOGIC against the strongest available baseline in each dataset–metric cell using the reported means in the revised manuscript. Ties denote equality at the displayed precision, not statistical equivalence; unavailable baseline results are excluded. Original Tables 6–9 are now numbered 7–10.

Revised tableMetric / datasetsValidation W/T/LTest W/T/L
1MRR / TKG3/0/14/0/0
2Hits@1 / TKG3/0/14/0/0
3MRR / THG4/0/04/0/0
4Hits@1 / THG4/0/04/0/0
7Hits@3 / TKG1/0/33/0/1
8Hits@10 / TKG1/0/32/0/2
9Hits@3 / THG3/1/03/1/0
10Hits@10 / THG2/1/12/1/1
Total32 cells per split21/2/9*26/2/4

RF-LOGIC achieves the highest reported mean in 26 of 32 test cells and 21 of 32 validation cells, supporting strong overall performance without implying superiority across every metric or split.

Q2. Per-dataset hyperparameters and selection. Table 6 reports:

Datasetλ\lambdaα\alphaSampled training negativesMk
Smallpedia10.120100500
Polecat20.120100500
ICEWS0.50.120100500
Wikidata00.1560300
Software0.250.120100500
Forum0.00010.0540004000
GitHub10.1280400
MyKet10.02600600

Here, k is the total per-entity history limit and M is the per-relation limit denoted m in Appendix B. The revised notation uses for decay and , for the two regularization coefficients in Eq. (7). Appendix B reports tuning both coefficients over {0,0.05,0.1,0.15,0.2} and selecting the best validation checkpoint.

Q3. Runtime, memory, and smaller histories. Appendix F adds the following measurements for RF-LOGIC, CEN, and RE-GCN:

DatasetModelTrain queries/s ↑Test queries/s ↑Peak GPU memory (GB) ↓
SmallpediaCEN22,67456820.8
SmallpediaRE-GCN38,8001,2389.6
SmallpediaRF-LOGIC1,1395203.8
PolecatCEN3,21118522.9
PolecatRE-GCN6,23343511.0
PolecatRF-LOGIC1,2327050.6
ICEWSCEN7,34233020.0
ICEWSRE-GCN12,8177679.5
ICEWSRF-LOGIC9303139.7

RF-LOGIC trains more slowly than both methods on these datasets. It has the highest inference throughput on Polecat and lower throughput on Smallpedia and ICEWS. Its peak GPU memory is lower on Smallpedia and Polecat, and comparable to RE-GCN on ICEWS (9.7 versus 9.5 GB). These measurements support dataset-dependent efficiency trade-offs. Point 6(c) gives the memory performance sensitivity results.

Q4. Identity of TGN-edge-type. Yes: it is the TGB 2.0 edge-type variant of TGN described in point 3, not a new baseline architecture introduced by us. Its definition and citation are now included in Section 4.1.

Q5. Forum metric provenance. The original error was combining published MRR with our rerun Hits values. The revised tables currently report the following means for our reruns:

ModelSplitMRRHits@1Hits@3Hits@10
TGNValidation0.5080.3000.6800.881
TGNTest0.3860.2290.6090.864
TGN-edge-typeValidation0.4900.3000.6800.881
TGN-edge-typeTest0.3940.2290.6090.864

Comments on the paper

1 comment
~Julia_Gastinger1OpenReview ↗

Question on "Model Update Memory" in the Code

Dear Authors.

I have been following the discussion with great interest, particularly because of the strong results on Polecat. For this reason I was also looking at the supplementary code. In this context I have a question on how the memory is updated during evaluation:

In test(), model.update_memory() seems to be called after every query. The paper states that memories are updated online in chronological order, so that “all memory entries correspond to events with timestamps strictly smaller than t+”. However, since many queries share the same timestamp, per-query updates would mean that a query at timestep t is scored with a memory that already contains other ground-truth facts from the same timestep t. This would include invrese edges, since TGB2.0 stores each fact together with its inverse at the same timestamp. If so, predictions for timestep t would use ground-truth information from t itself, which would amount to test-set leakage under the single-step forecasting protocol of TGB 2.0, where only facts from earlier timesteps may be used.

If I understand correctly, such facts from the same timestamp would then also be treaded as most recent evidence. I wonder if this is the reason for the large gains on the daily datasets like polecat and icews?

Could you please clarify whether my reading of the code is correct? If yes, would it be possible to report results where memory is updated only after all queries of a timestamp have been scored, to ensure that predictions for t only use information from earlier timestamps?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.