- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Open-Domain Forecasting on temporal knowledge graphs (TKGs), in which the model receives only the historical TKG and a forecast horizon , and must output a variable-sized set of complete future facts , without any query fields. The proposed method, ODCAST, consists of three stages: (I) generating initial hypotheses using prediction slots tied to temporal offsets; (II) constructing self-instantiating queries (SIQs) from the entities and relations currently predicted by the slots, then iteratively retrieving historical evidence under a global edge-retrieval budget and refining slot states; and (III) combining neural and repetition-based candidates, using two separate thresholds for repeated and novel facts to determine the output set. An independent logistic-regression trajectory expert also reranks repeated candidates, affecting average precision (AP) but not the output set.
Critical strengths
- The problem setting is practically meaningful. Figure 1 clearly illustrates the gap between conditional link prediction and real-world forecasting scenarios in which the model does not know what to ask about in advance.
- The evaluation protocol is comprehensive. The paper reports Strict F1, Soft F1, CardMAE.
- The data splits are rigorous. Table 3 reports the dataset and evaluation specifications.
Critical weaknesses
- The abstract and Contribution 1 claim that “we are the first to formulate the Open-Domain Forecasting problem on TKGs.” However, RE-NET [1] has already explored the generation of complete future events or graphs through multi-step reasoning procedure.
- Some time prediction baselines should also be included, such as GHT [2].
- In Table 1, the baselines have CardMAE values 3–30 times higher than that of ODCAST, and the reason remains unclear. The baselines are adapted using an author-designed method, but Appendix A.3 describes the selection of candidate caps and thresholds only in general terms, without providing specific values. Consequently, it is currently unclear whether ODCAST's advantage stems from better fact scoring, more effective output cardinality control, or simply weaker baseline adaptation.
- Missing pure repetition-heuristic baselines. Table 2 shows that removing recurrence-score fusion causes the largest drop in Strict F1, while Figure 12 shows that the model achieves its highest recall on frequently repeated triples. The paper does not include a corresponding learning-free repetition baseline in Table 1. For example, such a baseline could output triples that appeared at least times within the most recent days, assign their timestamps as , and tune on the development set. This baseline could be particularly strong on datasets such as ICEWS, which exhibit high repetition rates.
- The paper presents SIQ as core contribution for addressing CH1. However, Figure 3 reports only HSR and the number of retrieved edges, without examining how and affect Strict F1 or Exact AP. The ablation study in Table 2 also lacks a “w/o SIQ” or “” setting.
[1] Recurrent Event Network: Autoregressive Structure Inference over Temporal Knowledge Graphs
[2] Graph Hawkes Transformer for Extrapolated Reasoning on Temporal Knowledge Graphs
Questions for authors
- How much of the Strict F1 performance gap remains after equipping all baselines with the same oracle output number?
- How many independent full training runs do the “± SD” values represent (e.g., Figure 6)?