- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The data introduces the FoodTrade dataset, a dynamic network covering global trade for four staple grains, as an additional benchmark for temporal graph-based machine learning algorithms. The dynamic network includes additional node and edge level attributes which provide a richer network than existing trade-based networks.
An initial benchmarking for two regression problems show that otherwise SotA machine learning algorithms for dynamic networks struggle on the FoodTrade dataset, especially compared to a moving average. This is also observed for the trade network on the Temporal Graph Benchmark website so this result is not so surprising. The dynamic network is split into train/validation/test using rolling windows, which is consistent but may not work best for each of the tested models.
Critical strengths
New trade dataset highlighting the issues, even in smaller datasets, for SotA graph-based machine learning with additional node and edge level covariate information posing a newer challenge. It is important that research introducing novel datasets for benchmarking are introduced widely to focus research effort. I do not want my comments elsewhere to downplay the importance of work like this.
Critical weaknesses
As a benchmarking dataset, this does not offer much extra beyond existing dynamic trade datasets. For instance, Structure reducibility of mulitlayer networks, De Domenico et al. (2015) introduces a multilayer trade network over ten years. The Temporal Graph Benchmark website has a small trade example for a node classification task but with a only single edge weight, so a multilayer dynamic network is a natural comparison.
The most interesting addition are the node and edge covariate information, but an EDA for this information is lacking in the dataset description.
Questions for authors
- What were the motivation for including a rolling window for the train/validation/test setup? The regression can be done using the temporal sequence of network snapshots, so it felt like this approach restricted how machine learning algorithms could be applied to the data.
- Can you provide more details about the other possible tasks for this dataset beyond classification and prediction? The shock response simulation and nutrition classification felt like a stretch for what is possible using this dataset. If these are possible, then this would be a big selling point for the FoodTrade data.
3 comments
Thank you for your response.
Firstly, my W1 comment was supposed to say "over ten years ago" not "over ten years", so apologises for any confusion there. While the initial De Domenico et al. (2015) paper had a different aim, this dataset has been studied later as part of dynamic multilayer network analysis. It is not identical but the similarities are certainly enough to be cited.
I have reservations regarding forcing rolling windows for training/validation/test. I understand that this is commonly used, but by insisting on its inclusion in the benchmarking, this naturally restricts methods that can be tested. If the windowing is useful, then that can always be incorporated by a new technique, rather than being compulsory.
I am still not convinced that this dataset provides enough novelty and, while I hope I am wrong, I don't see it supplanting the TGB Trade benchmark. However, I may have underplayed it significance in bringing together a few existing dynamic multilayer networks into one place, so have improved my rating.
Regarding W1, we agree and have now cited De Domenico et al. (2015) as related work on multi-layer trade networks. We further clarify the rolling-window setup and address novelty concerns relative to TGB-Trade as follows.
-
Clarification on rolling windows
We would like to clarify that rolling windows for train/validation/test are not enforced by the dataset itself. FoodTrade remains accessible as a sequence of network snapshots, and users can define train/validation/test splits over arbitrary years and apply models directly to the full sequence. The rolling-window construction is instead part of the default benchmark protocol. It converts the relatively short annual sequence into multiple supervised forecasting instances and provides a common historical context across substantially different model classes. This is particularly useful in the low-sample regime of FoodTrade, while allowing each model to receive the same information in its required form, e.g., as a table, temporal sequence, or sequence of graphs. Thus, the benchmark provides a standardized comparison, without restricting future methods from using varying historical contexts.
-
Novelty relative to TGB-Trade
The goal of FoodTade is not to supplant the TGB Trade benchmark, but rather to complement it, as the two address different learning problems. TGB-Trade aggregates all agricultural products into a single monetary value network and evaluates next-year prediction of how a country's trade is distributed across partners as a ranking task. FoodTrade instead preserves crop-specific networks and evaluates quantitative prediction of country-level caloric supply and bilateral trade volumes, together with substantially richer node and edge covariates. The crop-trade domain additionally gives the benchmark meaningful downstream applications such as studying trade dependence, rerouting, and shock-response cascades.
Beyond the food trade domain, as a general discrete-time temporal graph-learning benchmark, evaluation across several model classes exposes where current temporal GNNs fail to exploit the available temporal, structural, and covariate information effectively. For example, on the Twitter Tennis benchmark [1], using two experimental settings [1,2], we find that a simple moving-average baseline outperforms their best reported temporal GNNs (MSE: 0.286 vs. 2.006 and 0.331 vs. 0.440, respectively). Such results highlight the need for challenging benchmarks that reveal these limitations and support the development of improved temporal GNN architectures and training strategies.
[1] PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models (CIKM 2021)
[2] Permutation Equivariant Neural Controlled Differential Equations for Dynamic Graph Representation Learning (NeurIPS 2025)
Rebuttal
We thank the reviewer for their insightful comments. We address their concerns below and would be happy to answer any further questions.
Inter-layer similarity analysis (W1)
The paper by De Domenico et al. (2015) and the FoodTrade benchmark have significantly different aims. The former quantifies inter-layer similarity using (unweighted) Jensen-Shannon (JS) distances and studies how a high number of layers (240 for the trade network) can be reduced to fewer but more distinctive ones. Our paper provides a benchmark for temporal graph learning with four layers of staple crops, which have potentially different trade dynamics. As two crop layers can have very similar binary topology yet differ significantly in trade-volume distributions or response to shocks, we evaluate FoodTrade inter-layer similarity by the Jaccard and weighted JS distances, which, between all pairs of crop layers across all years, are 0.75 and 0.85, respectively, marking high difference in the trade networks. Fig. 17 shows the JS distance at the crop-pair and year level. Additionally, De Domenico et al. (2015) use a trade network only for 2010, not over ten years. It is also not explicitly stated whether the edge weights are trade volumes or values, and whether the network is directed or undirected.
Covariate EDA (W2)
We divide node and edge covariates into categories and add them to the feature metadata. Fig. 18 provides an overview of the relationship across the high-dimensional feature space through within- and cross-category correlations. We also ablate the contribution of different categories to model performance in Table 19. Since the utility of features can vary across different tasks supported by the benchmark, we retain all the features at the dataset level, so purposeful feature selection and further feature engineering can be carried out as appropriately needed. Recent works have addressed feature selection particularly in the context of graph learning [1,2]. FoodTrade provides a new setting in which such methods can account for temporal roles and whether features are used locally or through message passing.
[1] Adaptive Node Feature Selection for Graph Neural Networks (ICML 2026)
[2] Let Your Features Tell The Differences: Understanding Graph Convolution By Feature Splitting (ICLR 2025)
Motivation for rolling windows (Q1)
Training/validation/test target years are chronologically separated to ensure evaluation on future unseen years, while a fixed context length provides a controlled comparison across model classes by giving each model access to the same amount of historical information. The evaluation framework then constructs this information as required by the model, e.g., a table, temporal sequence, or sequence of graphs (network snapshots). Rolling windows are a standard forecasting construction in time series forecasting [3,4]. including economic and trade domains. It is particularly useful here for converting a short temporal sequence into multiple supervised forecasting instances, while maintaining a common interface across vastly different model classes and enabling their direct comparison.
[3] Z-GCNETs: Time Zigzags at Graph Convolutional Networks for Time Series Forecasting (ICML 2021)
[4] Nowcasting trade in value added indicators. (OECD Statistics Working Papers 2023)
Higher-order tasks (Q2)
We agree that shock-response simulation and nutritional adequacy assessment are more complex than the node- and edge-level tasks evaluated here. These higher-order tasks are naturally compositional: they depend on accurate estimation of the quantities benchmarked in this work, so errors in basic forecasting would propagate into downstream simulations or classifications. For example, a trade cascade can arise when changes in domestic production or supply affect trade flows, subsequently altering supply in other countries and propagating through the network. Reliable simulation of such dynamics first requires accurate prediction of node- and edge-level quantities across varying network conditions. Otherwise, errors compounding over multiple steps can become difficult to diagnose. The present contribution of FoodTrade is to establish this benchmark foundation: a curated, attributed temporal graph dataset, standardized prediction tasks, and an extensive baseline evaluation of core quantities such as country-level supply and bilateral trade volumes. Establishing reliable performance on these building blocks is a necessary step toward more complex cascade and food-security tasks, which the dataset and evaluation framework are designed to support.