acceptodds
Under review as a conference paper at ICLR 2027

WHEN DOES RELATIONAL CONTEXT HELP PREDICT TECHNOLOGY ENTRY? EVIDENCE FROM U.S. METROPOLITAN AREA

Abstract

How does the inclusion of relational information improve predictions made us- ing history? We use U.S. metropolitan technology entry during 2000–2019 to construct a panel of 281,236 observations of 100 Core-Based Statistical Areas in- tegrating 541 CPC technology subclasses over the 10-year period. We investigate if relations with other cities forecast technology entry. We test if the benefit de- pends on how new the technology is relative to the city’s technology level. We use citation datasets to improve our forecast and measure the benefit against a baseline using the history and location of the focal city. The benefit is strongly dependent on technology novelty. For the most novel technology, the benefit is 0.016, and for the most familiar it is−0.002, for a difference of 0.018 (95% CI [0.002, 0.036]). The same effect is observed when using collaborations between inventors and when the same inventors work for the same company. As the col- laborations and company affiliations are similar—pairwise r = 0.81–0.97—we give greater weight to the results of the citation study and consider the effects of the collaborations and company affiliations as correlated views of a common diffusion signal rather than independent replications. Citation exposure, like ge- ography, predicts the outcome of interest. We consider the effect of relations, the intensity of relations, and other factors. Relational networks can be randomized, but the observed randomization achieves only Jaccard 0.645, so strong rela- tions and a “small world” effect constrain what can be learned. A graph neural network achieves forecasting accuracy of 0.6659. Conditional on technology nov- elty, we find the network captures relational information but the gradient is not significant; forecasts do not significantly vary from the baseline. The novelty of the technology may also condition the forecasts. The results jointly demonstrate that relational context enhances predictions pri- marily for technologically recent cases of competitive ent

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Temporal Graph Learning, Representation Learning, Relational Learning, Dynamic Graphs, Spatiotemporal Prediction, Transfer Learning, Graph Neural Networks, Technology Forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DhUGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies technology-entry forecasting in U.S. metropolitan areas using patent-derived relational networks. The central claim is that relational information becomes valuable primarily when predicting technologically novel entries. This hypothesis is evaluated through gradient-boosted models, statistical analyses, and temporal GNN baselines.

Critical strengths
  1. One aspect I liked is that the paper focuses on when relational information becomes useful, rather than simply whether it improves performance.

  2. The additional analyses (bootstrap confidence intervals, permutation tests, and novelty-based stratification) help support this claim beyond a simple comparison of predictive scores.

Critical weaknesses
  1. From an ICLR perspective, I found the main contribution to be empirical rather than methodological. The study is carefully executed, but it does not introduce a new learning framework or modeling approach.

  2. Although statistically significant, the reported improvements remain relatively small. The citation network improves performance by roughly +0.006 AP over the baseline, and even for the most novel technologies the gain reaches only about +0.016 AP. While these differences are measurable, it is not entirely clear whether they translate into meaningful practical benefits in real-world forecasting settings.

  3. While the GNN slightly outperforms the best feature-based model (+0.005 AP), the improvement is limited and does not provide clear evidence for the proposed novelty-related mechanism. A more detailed analysis of the GNN's predictions could help clarify whether the model actually exploits relational information for highly novel technologies.

  4. The analysis is restricted to the 100 most innovative U.S. metropolitan areas. As a result, it is not obvious whether the findings would also hold for smaller cities or less mature innovation ecosystems. While the authors acknowledge this limitation, I would like to see a deeper discussion of its implications.

Questions for authors
  1. How sensitive are the results concerning the novelty gradient to alternative definitions of technological similarity and novelty?
  2. Given the modest improvements in AP, do the authors have evidence that these gains would make a practical difference in technology-entry forecasting?
  3. Is it possible to separately quantify the contribution of the network structure versus that derived from simple aggregate measures of exposure?
  4. Can the authors explain why the GNN does not show a statistically significant novelty-related effect, despite having access to relational information?
  5. Would the conclusions remain valid if different forecasting horizons were used (e.g. 1 year or 5 years, instead of 3 years)?
Reviewer_2bMZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates whether inter-city ties help predict a metropolitan area’s entry into a specific CPC technology subclass within the next three years. The prediction target is defined as a cell with no patent in that subclass during the current year, though re-entries are permitted. Covering 100 U.S. CBSAs and 541 subclasses, the study augments local and geographic features with exposure measures derived from citations, co-inventors, or shared assignees. It then evaluates performance gains across quartiles of technological novelty.

Critical strengths
  1. The city sample and technology-similarity matrix are fixed using pre-period data, and model development is temporally separated from testing. These design choices are well-suited to the forecasting task.
  2. The authors test the Q4–Q1 contrast using city-level resampling and transparently acknowledge both the high correlations among network channels and the non-significant GNN novelty result. This restraint makes the primary citation finding easier to evaluate.
Critical weaknesses
  1. A filing-year cutoff may not reflect information available at the time. Features are drawn from a 2024 release of eventually granted patents, indexed by filing year. However, a patent filed in year t may only have been published, granted, classified, or cited later. The claim that all features were observable at t requires either an explicit audit of grant/publication dates or a reporting-lag experiment.
  2. The conclusions exceed what the small and inconsistently reported results support. The held-out analysis focuses on 2019, where citation exposure adds ~0.006 AP and the GNN beats the best tabular model by ~0.005 AP, yet neither comparison reports city-clustered confidence intervals in the main text. The Q4–Q1 interval barely excludes zero, and intermediate quartiles are non-monotonic, making the term “novelty gradient” stronger than the evidence warrants.
  3. The writing flow is difficult to follow.
Questions for authors
  1. Were patent and citation features filtered by the date they became publicly observable, rather than solely by filing year? How do results change under realistic grant or publication delays?
  2. What are the city-clustered confidence intervals for the pooled citation gain and the GNN–tabular difference? Does the Q4–Q1 contrast hold under prevalence-adjusted AP, lift, or ROC-AUC?
Reviewer_aZ5fOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates when relational context/relational exposure is useful for temporal prediction and examines the incremental predictive value of relational network features relative to local history and geographic exposure. The use of temporal leakage controls, quantile-based stratification, cluster-level bootstrap/permutation tests, and unit-weight ablation analyses reflects sound methodological design. However, I consider the paper's methodological contribution to machine learning relatively limited for an ICLR submission.

Critical strengths

S1. The paper does not simply ask whether relational features are useful, but rather under what target conditions they are useful—that is, whether the incremental value of relational context varies with target novelty. This is a more interesting question than simply demonstrating an improvement in overall AP.

S2. The paper evaluates ΔAP across novelty quartiles and uses city-clustered bootstrap and within-CBSA permutation tests to assess the Q4–Q1 difference, reporting 95% confidence intervals and permutation p-values. It also conducts analyses of cross-channel correlations, unit-weight ablations, and degree-preserving randomization. Overall, the evaluation is comprehensive.

Critical weaknesses

W1. The paper's main contribution lies more in applied empirical analysis than in methodological innovation in machine learning. The GNN component employs a heterogeneous GNN with two rounds of message passing and 32-dimensional embeddings, which constitute a fairly standard architecture.

W2. The core claim is that the incremental predictive value of relational exposure increases with novelty. The supporting evidence is primarily based on the difference in AP gains between Q4 and Q1, i.e., ΔAP(Q4) − ΔAP(Q1). However, the absolute gains are relatively small: the pooled citation-network increment is +0.006 AP, while the increment in Q4 is +0.016 AP. Moreover, the baseline entry rates differ substantially across novelty quartiles (47.8% in Q1 versus 13.6% in Q4). I recommend supplementing the evaluation with precision–recall curves, Recall@K, decision curve analysis, and an assessment of practical utility under a fixed prediction budget.

W3. The reproducibility statement only provides the data sources and DOI, but lacks a code repository link, feature construction scripts, hyperparameter search ranges, random seeds, details of GNN training, and computational resource specifications.

W4. The paper has a relatively limited set of references and provides insufficient discussion of temporal GNNs, graph representation learning, conditional predictive validity, and selective prediction/calibration.

Questions for authors
  1. Could the authors report an interaction model that treats novelty as a continuous variable, rather than relying solely on quartile-based stratification? This would help assess whether the observed gradient is monotonic and robust.

  2. Could the authors report precision and recall under a fixed prediction budget, along with calibration metrics and decision curve analysis? Given the relatively small improvements in AP, more direct evidence of the practical decision-making value of relational features is needed.

  3. The novelty quartile boundaries are computed on the training set, while the novelty measure itself evolves over time as the technology portfolio expands with tt. Could the authors report year-to-year shifts in the novelty distribution and assess the sensitivity of the results to the choice of quartile boundaries?

  4. Could the authors present the rolling-origin results for 2017–2019 in the main paper or a supplementary table, reporting both the pooled estimate and year-to-year heterogeneity?

  5. The authors could report the rolling-origin results for 2017–2019 in the main paper or a supplementary table, including both the pooled estimate and year-to-year heterogeneity.

  6. The paper should provide a complete anonymized code repository, specify all random seeds, report the GNN hyperparameters, and document the computational environment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.