acceptodds
Under review as a conference paper at ICLR 2027

Beyond Predefined Queries: Open-Domain Forecasting on Temporal Knowledge Graphs

Abstract

Forecasting future interactions and relationships has broad applications in domains such as supply chains, corporate activities, and international relations. Temporal knowledge graphs (TKGs) represent time-dependent relational facts, but conventional TKG forecasting typically predicts a missing field in a predefined future query. For example, a model may predict which company a given supplier will supply on a specified future date, while the supplier, relation, and timestamp are provided. In many applications, these fields are unknown and must be predicted together. To address this gap, we are the first to formulate the Open-Domain Forecasting problem on TKGs and propose an Open-Domain forecasting (ODCAST) framework to address it. ODCAST predicts a variable-size set of complete future relational facts from a historical TKG and a forecast horizon without predefined target queries. ODCAST uses prediction slots to form initial hypotheses and Self-Instantiated Queries to retrieve historical evidence under a shared graph-access budget. A set decoder combines neural proposals with recurrence modeling and selects complete quadruples through branch-specific thresholds, while an independent temporal trajectory expert reranks recurrent candidates. Experiments on three benchmark datasets show that ODCAST consistently outperforms the best-performing baseline in Strict F1 across all evaluated forecast horizons, achieving improvements of up to 43.15%, 110.93%, and 25.04% on ICEWS14, ICEWS05–15, and tkgl-polecat, respectively.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Open-Domain Forecasting, Temporal Knowledge Graph Reasoning, Graph Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mS2QOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Open-Domain Forecasting on temporal knowledge graphs (TKGs), in which the model receives only the historical TKG and a forecast horizon HH, and must output a variable-sized set of complete future facts {(s,r,o,t)}\{(s,r,o,t)\}, without any query fields. The proposed method, ODCAST, consists of three stages: (I) generating initial hypotheses using prediction slots tied to temporal offsets; (II) constructing self-instantiating queries (SIQs) from the entities and relations currently predicted by the slots, then iteratively retrieving historical evidence under a global edge-retrieval budget BB and refining slot states; and (III) combining neural and repetition-based candidates, using two separate thresholds for repeated and novel facts to determine the output set. An independent logistic-regression trajectory expert also reranks repeated candidates, affecting average precision (AP) but not the output set.

Critical strengths
  1. The problem setting is practically meaningful. Figure 1 clearly illustrates the gap between conditional link prediction and real-world forecasting scenarios in which the model does not know what to ask about in advance.
  2. The evaluation protocol is comprehensive. The paper reports Strict F1, Soft F1, CardMAE.
  3. The data splits are rigorous. Table 3 reports the dataset and evaluation specifications.
Critical weaknesses
  1. The abstract and Contribution 1 claim that “we are the first to formulate the Open-Domain Forecasting problem on TKGs.” However, RE-NET [1] has already explored the generation of complete future events or graphs through multi-step reasoning procedure.
  2. Some time prediction baselines should also be included, such as GHT [2].
  3. In Table 1, the baselines have CardMAE values 3–30 times higher than that of ODCAST, and the reason remains unclear. The baselines are adapted using an author-designed method, but Appendix A.3 describes the selection of candidate caps and thresholds only in general terms, without providing specific values. Consequently, it is currently unclear whether ODCAST's advantage stems from better fact scoring, more effective output cardinality control, or simply weaker baseline adaptation.
  4. Missing pure repetition-heuristic baselines. Table 2 shows that removing recurrence-score fusion causes the largest drop in Strict F1, while Figure 12 shows that the model achieves its highest recall on frequently repeated triples. The paper does not include a corresponding learning-free repetition baseline in Table 1. For example, such a baseline could output triples that appeared at least mm times within the most recent WW days, assign their timestamps as t0+δtt_0+\delta t, and tune mm on the development set. This baseline could be particularly strong on datasets such as ICEWS, which exhibit high repetition rates.
  5. The paper presents SIQ as core contribution for addressing CH1. However, Figure 3 reports only HSR and the number of retrieved edges, without examining how KK and BB affect Strict F1 or Exact AP. The ablation study in Table 2 also lacks a “w/o SIQ” or “K=0K = 0” setting.

[1] Recurrent Event Network: Autoregressive Structure Inference over Temporal Knowledge Graphs

[2] Graph Hawkes Transformer for Extrapolated Reasoning on Temporal Knowledge Graphs

Questions for authors
  1. How much of the Strict F1 performance gap remains after equipping all baselines with the same oracle output number?
  2. How many independent full training runs do the “± SD” values represent (e.g., Figure 6)?
Reviewer_9VfhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Unlike query-completion TKG forecasting, open-domain forecasting predicts an unconstrained, variable-sized set of complete future facts (s, r, o, t) given only the historical graph and the forecast horizon. The authors propose a three-stage framework, ODCAST:

  • During the Initial Prediction, it encodes history into horizon-scaled slots to decode initial hypothesis tuples.
  • Then, at Iterative Query Instantiation, it generates uncertainty-prioritized Self-Instantiated Queries (SIQs) to retrieve historical evidence under a fixed graph-access budget.
  • At last, the Set Materialization fuses neural and recurrent proposals, applies trajectory reranking, and filters facts using branch-specific thresholds.

The framework is evaluated on ICEWS14, ICEWS05–15, and tkgl-polecat; ODCAST consistently outperforms five adapted baselines in Strict and Soft F1 across all forecast horizons.

Critical strengths
  1. The paper introduces a novel and realistic problem formulation that addresses the gap between synthetic query completion and practical open-ended event forecasting.

  2. The three-stage ODCAST framework directly tackles the exponential combinatorial space of quadruples.

  3. The paper establishes a comprehensive benchmark protocol using well-grounded metrics (Strict F1, Soft F1 using bipartite matching, CardMAE, and Exact AP) and extensive ablations.

Critical weaknesses
  1. The baseline performance is bounded by the quality of the heuristic prefix generator.

  2. On the largest dataset (tkgl-polecat, containing >150k entities), ODCAST achieves higher Strict/Soft F1 but yields a higher CardMAE than NADEx (e.g., 342.07 vs. 317.95 at H=1). This indicates that in extremely large, sparse entity spaces, branch threshold calibration can struggle to accurately regulate output cardinality.

3.The iterative SIQ mechanism requires multiple rounds of attention over retrieved historical frontiers.

Questions for authors
  1. How much does the performance of the adapted baselines depend on the specific prefix generator used?
  2. What causes the higher CardMAE on tkgl-polecat compared to NADEx despite ODCAST's higher F1?
  3. Does the fixed retrieval budget become a bottleneck as H increases, and did you test horizon-scaled access budgets?
Reviewer_qRSnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies temporal knowledge graph forecasting when no part of a future query is given. From the observed graph and a forecast horizon, the model has to decide which events will occur, when they will occur, and how many events to predict. ODCAST starts with a set of event hypotheses, uses them to query the historical graph, and combines neural proposals with candidates drawn from recurring events. The authors evaluate the resulting event sets on three datasets. ODCAST achieves the highest reported Strict F1 among the compared methods at all three forecast horizons on each dataset.

Critical strengths
  1. The task captures a limitation of the usual forecasting setup. Giving a model the subject, relation, and date of a future event already tells it quite a bit about what will happen. Removing that information changes the problem in a meaningful way. In particular, a model can rank the correct object highly once prompted and still have little ability to anticipate that the event will happen at all. The task definition and exact-event evaluation make this distinction clear.

  2. The paper looks at where predictions are lost. I found the candidate-coverage breakdown in Figure 11 particularly informative. If a gold event never enters either candidate pool, changing the final threshold cannot recover it. Read alongside the separate results for recurrent and novel events in Figure 9, this gives a more informative picture of the model than the aggregate F1 alone. The gains in Table 1 are also consistent across datasets and horizons, even though event-count error does not improve consistently on tkgl-polecat.

Critical weaknesses
  1. The contribution of self-instantiated queries to forecast accuracy remains unclear. Figure 3 shows that a larger retrieval budget helps recover more historical support. The question I was left with is whether the retrieved evidence helps the model predict the right future events. DirectSet removes several components at once, and Table 2 does not report the identity-update reference described in Section 4.2. I also could not tell from the supplement whether the Full ODCAST results in Table 1 include the evidence-driven update at test time. A comparison against that reference, with the same retrieval budget and final selection procedure, would help establish how much this part of the method contributes to Strict F1.

  2. The evaluation leaves an important recurrence baseline unreported. On ICEWS14 (at H=1), removing recurrence-score fusion lowers Strict F1 from 14.38 to 10.06. Given the size of that drop, I would have liked to see how far a simple forecaster can get by predicting recent or frequent events and calibrating how many to emit. The supplement includes such baselines, but the paper does not report their scores. The comparison with the adapted TKG models raises a related question. These models rely on a generator that supplies candidate prefixes. For TiRGN, CognTKE, and NADEx, I could not find the corresponding adaptations in the supplement. Without knowing how often their candidate prefixes cover the ground-truth events, it is difficult to interpret the gap in Table 1.

Questions for authors
  1. Was the evidence-driven update active at test time for the Full ODCAST results in Table 1? For one representative setting, such as ICEWS14 (at H=1), could you report Strict F1 for Full ODCAST and the identity-update reference under the same retrieval budget and selection procedure? Please indicate whether the reference was trained separately or evaluated by disabling the update at inference.

  2. How do the frequency and recency-frequency baselines perform under the same history cutoff and calibration rules as ODCAST? Strict F1 and the predicted versus observed event counts for the same setting would help interpret the main result.

  3. How were candidate prefixes generated for TiRGN, CognTKE, and NADEx? What fraction of ground-truth events had a matching prefix before the models ranked candidate objects?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.