acceptodds
Under review as a conference paper at ICLR 2027

Learning to Forecast on Multiple Spatiotemporal Hierarchies

Abstract

Hierarchical forecasting exploits aggregation structures that expose multiple views of the same underlying spatiotemporal forecasting problem, while ensuring coherence among forecasts. Individual series provide fine-grained information, while progressively coarser aggregates reveal temporal and spatial patterns at different scales. In the associated deep learning architectures, the hierarchy determines which aggregate series are available and how they are processed together with individual series (e.g., through reconciliation or message-passing operators). Different hierarchies thus provide distinct learning signals and inductive biases for forecasting the same collection of time series. Nonetheless, existing architectures are typically trained on a single, often fixed, structure. In this work, we propose a graph deep learning framework that leverages variations across hierarchies as multi-task regularization. We show that jointly learning across these structures can reduce over-specialization to any single hierarchy and improve forecasting performance. Indeed, training across hierarchies provides an inductive bias toward representations that are useful across different structures, and as such, generalize better. Making forecasts coherent with the aggregation constraints for each sampled hierarchy is computationally expensive: to complement the framework, we propose a novel operator—based on a top-down approach—that produces coherent forecasts with linear time and space complexity w.r.t. the hierarchy size. Experiments show that our approach improves forecasting accuracy over training on a single structure and is competitive with current state-of-the-art methods.

Primary area:
learning on time series and dynamical systems
Keywords:
spatiotemporal forecasting, time series, reconciliation, graph neural networks, hierarchical forecasting

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_8PkoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SHR, a graph deep learning framework that leverages variations across hierarchies as multi-task regularization. Specifically, a training framework is used to optimize a shared spatiotemporal forecasting model across multiple structurally diverse hierarchies. In addition, an efficient recursive procedure is introduced to incorporate forecasts from every aggregation level while enforcing the corresponding coherence constraints. Experimental results demonstrate the effectiveness of the proposed methods.

Critical strengths
  1. The empirical analysis examines several relevant aspects of the proposed mechanism.

  2. The paper is easy to follow.

Critical weaknesses

The methodological novelty is limited. The forecasting architecture is adapted from HiGP, and forecast averaging across multiple hierarchies has been explored previously. The paper should more clearly establish the contribution of shared multi-hierarchy training beyond structural data augmentation and forecast combination.

The experimental comparisons leave several important issues unresolved:

  1. The main results average predictions across 20 hierarchies for SHR, whereas the fixed-hierarchy baseline uses one structure. Although the appendix includes evaluations without averaging, the headline comparisons do not clearly separate training regularization from additional inference computation.
  2. The scalability experiments primarily compare the proposed operator with dense projection-based reconciliation. Comparisons with implementations that exploit tree structure or sparse operations are needed.
  3. The inductive experiment compares SHR only with a fixed-hierarchy model on one GLA spatial split, providing limited evidence of its advantage over other transferable forecasting models.
Questions for authors
  1. How much of the improvement in Table 1 is retained when SHR uses a single inference hierarchy? Could you report comparisons under identical reconciliation and inference settings to isolate the contribution of multi-hierarchy training?

  2. The improvements reported in Table 1 over the baselines appear marginal, making it difficult to rule out the effect of random experimental variation. Could you report results averaged over multiple runs with different random seeds, together with standard deviations or statistical significance tests?

  3. The main architecture figure is somewhat difficult to interpret, and the interactions among the different modules are not fully clear. Could the authors provide a more detailed walkthrough of the pipeline, clarifying the role of each component and how they interact within the overall framework?

  4. Please evaluate the robustness of SHR under more challenging conditions. For example, does the method remain effective in long-term forecasting settings (e.g., T = 96, 192, 336, and 720)? Such experiments would help demonstrate the robustness and generalization ability of the proposed method.

6 comments

We thank the reviewer for appreciating our empirical analysis and for finding the paper easy to follow.

Weakness 1.1: The forecasting architecture is adapted from HiGP, and forecast averaging across multiple hierarchies has been explored previously.

The reference architecture stems from HiGP: please note that we are not interested in proposing a specific architecture, but rather a general framework, which we instantiate with this architecture. In fact, relying on an existing architecture allows us to isolate the effect of training on multiple hierarchies from architectural choices. As for “forecast averaging”, it is a very broad idea, and the main difference between our approach and existing methods is in how the forecasts being combined are obtained. In our case, forecasts are obtained from the same model by processing different hierarchical views of the same input data: this is a novel approach as we discuss in the paper.

Weakness 1.2: The paper should more clearly establish the contribution of shared multi-hierarchy training beyond structural data augmentation and forecast combination.

Our main contribution is introducing multi-hierarchy training and inference as a method for regularizing hierarchical spatiotemporal models and, as such, obtaining more accurate predictions in this context. This is a clear, distinct, and novel contribution with respect to prior works, as we discuss in the paper (Sec. 4). (1) Unlike data augmentation, this is not just about training on more data: the hierarchy has a strong impact not only on prediction targets but also on how the input is processed, as it defines the aggregates, the paths through which information propagates, and the coherence constraints. (2) As discussed above, while combining predictions obtained by different means is a standard approach to improve forecasting accuracy, the novelty here is how these forecasts are obtained. We have revised the introduction to make these aspects clearer.

Weakness 2: [...] Although the appendix includes evaluations without averaging, the headline comparisons do not clearly separate training regularization from additional inference computation. Question 1: How much of the improvement in Table 1 is retained when SHR uses a single inference hierarchy? Could you report comparisons under identical reconciliation and inference settings to isolate the contribution of multi-hierarchy training?

In SHR, the prediction is defined as the expected forecast over the distribution of hierarchies, which we estimate by averaging over sampled hierarchies (Eq. 6); which is why we report it in the main results. As you correctly point out, the appendix already included many analyses showing the impact of the regularization even without inference-time averaging. We have now moved one of these to the main body of the paper (Fig. 5), which compares, under identical reconciliation and inference settings, a model trained on a single fixed hierarchy with models trained on increasingly many and diverse hierarchies, all evaluated on the same hierarchies without averaging.

To further quantify how much of the improvement is retained without averaging, we ran an additional experiment in Tab.8 (App. C.1), where SHR is evaluated without inference-time averaging on two different fixed hierarchies. Results show gains in performance even when using a single completely random hierarchy from the training distribution. We expect even better results to be possible with better heuristics to choose the inference hierarchy (i.e., by choosing the hierarchy that performs best on the validation set). This provides further convincing evidence that performance improvements do not stem from inference-time averaging alone.

We thank the reviewer for pointing this out. We hope these additional results fully address the concern, and we are happy to continue the discussion.

Weakness 3: The scalability experiments primarily compare the proposed operator with dense projection-based reconciliation [...]

Our scalability experiment (Fig. 6) compares our approach against the one used in popular deep hierarchical forecasting models [1,2]. This involves computing the inverse of a matrix that is typically dense even if the initial matrices are sparse. More efficient implementations exist that exploit the tree structure or the sparsity of the hierarchy [3]. These implementations reduce the cost of projection-based reconciliation, but they still require solving a least-squares problem over the entire hierarchy. Our operator, instead, relies only on local parent-child operations, without any matrix inversions. We revised the discussion in Sec. 5, clarifying that the computational cost we refer to is that of standard dense implementations, and we are looking into adding another baseline in the revision.

Weakness 4: The inductive experiment compares SHR only with a fixed-hierarchy model on one GLA spatial split, providing limited evidence of its advantage over other transferable forecasting models.

The inductive experiment complements the other experiments presented in the paper, which provide extensive evidence of the effectiveness of the methods. The purpose of this experiment is to isolate the effect of multi-hierarchy training when the model is applied to unseen series, which is why we compare SHR with the same model trained on a fixed hierarchy. HiGP cannot be applied directly, as it relies on node embeddings that do not transfer to unseen graphs (as explained in Sec. 5). The original inductive results use a single region of LargeST, Greater Los Angeles (GLA), split into subgraphs. To strengthen this evidence, we added to Tab.4 a harder inductive protocol that uses the three geographically disjoint regions of LargeST: the model is trained on GLA and the Greater Bay Area and tested on San Diego, a region never seen during training. SHR (rand.) achieves the lowest error in both settings (Tab. 4) and at all horizons (Tab. 10), supporting the benefit of our approach. We thank the reviewer for this suggestion.

Question 2: The improvements reported in Table 1 over the baselines appear marginal, making it difficult to rule out the effect of random experimental variation. Could you report results averaged over multiple runs with different random seeds, together with standard deviations or statistical significance tests?

Table 1 already provides standard deviations over multiple runs with different random seeds, as do all other results in the paper. We now state it also in the caption of Tab. 1.

Question 3: The main architecture figure is somewhat difficult to interpret [...] Could the authors provide a more detailed walkthrough of the pipeline, clarifying the role of each component and how they interact within the overall framework?

We thank the reviewer for the suggestion. We have replaced Figure 1 with a more detailed walkthrough of the pipeline (see Fig.1), which separates the training and inference steps and shows the role of each component. At each training step, a hierarchy is sampled, the shared forecaster predicts all its nodes, the recursive operator makes the predictions coherent, and the loss is computed at every level. At inference, the same model is applied under several sampled hierarchies, and the coherent bottom-level forecasts are averaged. The same steps are described in Sec. 3.1 and at the beginning of Sec. 3.

Question 4: Please evaluate the robustness of SHR under more challenging conditions. For example, does the method remain effective in long-term forecasting settings (e.g., T = 96, 192, 336, and 720)? Such experiments would help demonstrate the robustness and generalization ability of the proposed method.

We did not consider long-term horizons because our method relies on spatial hierarchies, whose main benefit is to capture dependencies among the series, and these are most relevant at short horizons. For example, in traffic, long-horizon forecasts are largely driven by daily and weekly patterns, which reduces the advantage of exploiting a graph structure. For this reason, we follow the standard horizons of the spatiotemporal forecasting literature for that domain (Sec. 5). A natural way to address long-term forecasting would be to use temporal hierarchies, which aggregate the series over increasingly coarser time scales, and we mention this as an interesting extension of our work in Sec. 6. Regarding the robustness of our method, the paper already includes extensive experiments, including the inductive setting on unseen graphs (Sec. 5 and App. C.4), ablations (Tab. 2 and 3), and sensitivity analyses (App. C.2) that probe the effectiveness of the approach under many different scenarios.

We thank the reviewer for the feedback. We have addressed the points raised in the review, adding new experiments and extending the discussion where needed. We kindly ask the reviewer to reconsider the score in light of these clarifications, and we are happy to continue the discussion if any aspect remains unclear.

[1] Cini A. et al., Graph-based time series clustering for end-to-end hierarchical forecasting, ICML 2024

[2] Rangapuram S. et al., End-to-End Learning of Coherent Probabilistic Forecasts for Hierarchical Time Series, ICML 2021

[3] Hyndman R. J. et al., Fast computation of reconciled forecasts for hierarchical and grouped time series, Computational Statistics & Data Analysis 2016

Thank you for your response. While some of my concerns have been addressed, several key issues remain unresolved, including the need for stronger evidence of the proposed method’s advantages over other transferable forecasting models and an evaluation of SHR’s robustness under more challenging conditions. After considering the authors’ overall response, I have decided to maintain my original score.

Reviewer_WQXYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents a novel approach for spatiotemporal forecasting. Given a hierarchy on a spatiotemporal forecasting problem, the goal is to make coherent forecasts for the whole hierarchy. The temporal data at each node in the hierarchy is the average of the leaf node for that node in the hierarchy tree.

The main forecasting architecture in a graph neural network based model to make a forecast at each node. The forecasts are not coherent, and hence a method for obtaining forecasts coherently is proposed. The paper also presents a novel training approach where the model is trained a distribution on hierarchies, thus letting it generalize to new hierarchies.

Experimental results on benchmark datasets show that training on a distribution of hierarchies help with generalization.

Critical strengths

The paper presents an interesting idea of training a spatio temporal forecasting model which can make predictions for any general hierarchy on a dataset, useful when one has to predict on unseen hierarchies. Evaluation on benchmark datasets show improved prediction accuracy due to better generalization. The hypothesis is that, sampling multiple hierarchies during training effectively leads to more training data-points helping with generalization.

Critical weaknesses

The framing of the paper is not entirely clear. The datasets used in the paper are not the ones typically used for hierarchical forecasting evaluation. They are standard spatio-temporal forecasting datasets. A major question is if the proposed method designed for spatio temporal forecasting or hierarchical forecasting?

Several common hierarchical forecasting datasets are missing:

  • Australian Tourism / TourismLarge.
  • M5 retail sales.
  • Electricity demand.
  • Australian Labour.
  • Wikipedia page views / Wiki2.

Several related literature on hierarchical forecasting are also missing:

Hierarchy-aware deep forecasting / end-to-end hierarchical models

Hierarchy/coherence incorporated directly into training

Probabilistic coherent forecasting / coherence by construction

ML / nonlinear forecast reconciliation

Questions for authors
  • Is the paper scoped to forecasting problems without a specific hierarchy? That is, the hierarchy is not fixed and can be generated during inference?
  • Is the only purpose of the hierarchy to produce accurate bottom level forecasts. That is, the forecasts at the internal nodes are thrown away?
  • If the goal is to predict accurate forecasts for the leaf node, isn't using a full hierarchy tree with multiple levels + reconciliation, an overkill? One can get additional training datapoints using a simpler strategy: why not simply group related/close nodes (sample this grouping randomly), aggregate them and use them to expand the training dataset.
  • The utility of the reconciliation step is not clear in this context.
4 comments

We thank the reviewer for finding our idea interesting and for recognizing that training on multiple hierarchies improves generalization.

Weakness 1: The datasets used in the paper are not the ones typically used for hierarchical forecasting evaluation. They are standard spatio-temporal forecasting datasets. A major question is if the proposed method designed for spatio temporal forecasting or hierarchical forecasting?

Our target problem is spatiotemporal forecasting. We build on techniques from the hierarchical forecasting literature, such as aggregation structures and coherent forecasts, to design our method. Note that hierarchical representations are also used in spatiotemporal forecasting [1,2,4,5], even though in most cases no predefined hierarchy is available. This makes our setting quite different from those considered in standard hierarchical forecasting literature, which typically assume a given hierarchy and does not focus on spatiotemporal data (hence, the different benchmarks/datasets). We clarify this in the revised version of the paper.

Weakness 2: Several related literature on hierarchical forecasting are also missing

We agree that expanding the discussion of related work in hierarchical forecasting would be beneficial (we kept it condensed for space reasons and focused on the works most related to our setting). Following the reviewer's suggestion, we used the additional space in the revised version to expand this discussion in Sec. 4, including all the references suggested by the reviewer.

Question 1: Is the paper scoped to forecasting problems without a specific hierarchy? That is, the hierarchy is not fixed and can be generated during inference?

Yes, we focus on problems where there is no pre-defined target hierarchy. Nevertheless, we argue that training on multiple hierarchies can be beneficial (as a regularization) also when one or more target hierarchies exist, as training on more structures improves performance (as we show in our experiments).

To further assess this, we added an additional experiment (Tab.9 - App. C.1) where we consider two target hierarchies: a k-MIS hierarchy drawn from the distribution described in Sec. 3.2, and a hierarchy outside of SHR’s training distribution obtained through spectral clustering. For each of these two target hierarchies, we train a separate specialized fixed-hierarchy model and compare it against SHR. Note that this is an adversarial setting for SHR: the ‘Fixed Hier.’ baselines are specifically trained on the target hierarchy while SHR is agnostic to the target. Results show that SHR achieves errors lower than or comparable to those of the 'Fixed hier.' baselines on both hierarchies, across all datasets and aggregation levels. Hence, results show that when a target hierarchy is given, even only at inference time, the same SHR model can be applied directly to it and obtain accurate predictions without any fine-tuning.

Question 2: Is the only purpose of the hierarchy to produce accurate bottom level forecasts. That is, the forecasts at the internal nodes are thrown away?

No. While the paper mainly focuses on the bottom level, the forecasts at the internal nodes are coherent forecasts that can also be used, depending on whether they are relevant for the application. Training on multiple hierarchies improves them as well. The experiments in Fig. 4 and 8 directly assess this. The results indeed show that the error at the upper levels decreases as the training hierarchies become more diverse.

The additional experiment (Tab. 9 - App. C.1), which we discussed above, provides further evidence. At the intermediate and top levels, SHR achieves errors lower than or comparable to those of a model trained on the same hierarchy in most settings, including on the spectral hierarchy, which is outside of SHR's training distribution.

Question 3: If the goal is to predict accurate forecasts for the leaf node, isn't using a full hierarchy tree with multiple levels + reconciliation, an overkill? One can get additional training datapoints using a simpler strategy: why not simply group related/close nodes (sample this grouping randomly), aggregate them and use them to expand the training dataset.

As mentioned above, forecasting the bottom level isn't necessarily the only objective. Even when this is the case, using hierarchical architectures has been consistently shown to result in improved performance even with a fixed hierarchy [1,2,4,5] (this can be explained by several factors, including better architectural biases and the ability to capture multi-scale dynamics). Our empirical results confirm this (see, e.g., Tab. 1, where SHR outperforms or is comparable to the non-hierarchical baselines on all four datasets).

The training strategy the reviewer is suggesting is equivalent to training on a flattened sub-sample of raw time series and aggregates. The strategy would be appropriate in settings with no spatial structure, but in a spatiotemporal forecasting problem, it would discard the dependencies among time series one is trying to model (with no propagation mechanism, each time series would be processed independently from the others).

Question 4: The utility of the reconciliation step is not clear in this context.

Reconciliation acts as an additional regularization that works well in deep hierarchical forecasting models [1, 3] and can improve accuracy even when only bottom-level forecasts are of interest. In other words, reconciliation is a way of imposing meaningful constraints on the predictions that can improve prediction accuracy at each level of the hierarchy, e.g., see the theoretical analysis in [7] and the empirical study in [6], which focuses on a setting with no predefined target hierarchy. Moreover, with our recursive operator, aggregate forecasts directly contribute to the bottom-level predictions (Sec. 3.4), and we designed this step to be computationally manageable when training on many hierarchies. We appreciate the question; we will make this motivation more explicit in the paper.

We thank the reviewer for having raised interesting points, which helped us improve the work. We hope we have addressed all the reviewer’s questions. We are happy to continue the discussion if any aspect remains unclear. Otherwise, we would appreciate if the reviewer would consider increasing the score.

[1] Cini A. et al., Graph-based time series clustering for end-to-end hierarchical forecasting, ICML 2024

[2] Chen L. et al., Group-Aware Graph Neural Network for Nationwide City Air Quality Forecasting, 2023 TKDD

[3] Rangapuram S. et al., End-to-End Learning of Coherent Probabilistic Forecasts for Hierarchical Time Series, ICML 2021

[4] Yu B., et al., ST-UNet: A Spatio-Temporal U-Network for Graph-structured Time Series Modeling, 2021

[5] Hermes L., et al., A Graph-based U-Net Model for Predicting Traffic in unseen Cities, IJCNN 2022

[6] Zhang B. et al., Constructing hierarchical time series through clustering: Is there an optimal way for forecasting?, IJF 2025

[7] Di Fonzo T. and Girolimetto D., Forecast combination-based forecast reconciliation: Insights and extensions, 2024 IJF

Thank for the response and thanks for clarifying that the paper is about spatio-temporal forecasting and not hierarchical forecasting. In that case, it is still not clear why hierarchical reconciliation is useful for spatio-temporal forecasting. It doesn't seem related to the main idea in the paper. Could explain a bit more on use cases where hierarchical forecasting will be useful?

In that case, it is still not clear why hierarchical reconciliation is useful for spatio-temporal forecasting. It doesn't seem related to the main idea in the paper.

The main idea of forecast reconciliation is to use a known constraint (forecasts at different levels should add up) to regularize predictions and, as such, improve accuracy. This is relevant independently of the setting as it can improve performance across all the levels of the hierarchy, including the leaves. This is confirmed by results both in hierarchical [3] and spatiotemporal forecasting [1]. With this in mind, although not our main focus, a reconciliation operator is a core component of a hierarchical forecasting framework, also in the spatiotemporal forecasting context, and, in our case, contributes to regularizing predictions across sampled hierarchies. The approach that we design has the specific objective of keeping computational costs contained without giving up this additional source of regularization.

Could explain a bit more on use cases where hierarchical forecasting will be useful?

As we discussed, ideas from hierarchical forecasting are useful in spatiotemporal data processing as hierarchical representations allow for processing data at different granularities, which can improve forecasting accuracy even at the level of the single time series (this is confirmed by our results and by several related works). Moreover, there are many applications in spatiotemporal forecasting where having predictions at different scales (e.g., single sensor, home, city, region, nation, …) is useful on its own. These include forecasts of electricity demand [8,10], solar power production [9], road traffic [11], and more. In each of these applications, different use-cases can indeed require predictions at different levels of aggregation. For example, in electricity demand forecasting, predictions at different levels are needed for different purposes: fine-grained predictions allow for optimizing control of the power grid while forecasting coarser-grained aggregates is critical for optimizing storage and supply. Results in Figures 4 and 8, plus the new experiments we added for the rebuttal (Table 9), show that SHR can improve forecasting accuracy also at the aggregate levels of different hierarchies, providing high flexibility and accuracy in this context.

We hope this helps. Please let us know if any other doubts are preventing you from recommending acceptance. We will make these points clearer in the revision of the paper.

[1] Cini A. et al., Graph-based time series clustering for end-to-end hierarchical forecasting, ICML 2024

[3] Rangapuram S. et al., End-to-End Learning of Coherent Probabilistic Forecasts for Hierarchical Time Series, ICML 2021

[8] Taieb et S. B. al., Hierarchical Probabilistic Forecasting of Electricity Demand With Smart Meter Data, 2021 Journal of the American Statistical Association

[9] Yang D. et al., Reconciling solar forecasts: Geographical hierarchy, 2017 Solar Energy

[10] Pang Y. et al., Hierarchical electricity time series prediction with cluster analysis and sparse penalty, 2022 Pattern Recognition

[11] Benarmas R. B. et al., A deep learning hierarchical approach to road traffic forecasting, 2024 Journal of Forecasting

Reviewer_BqodOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The topic of the paper is a multi-scale GNN, which acts over a hierarchical graph. The authors propose a type of multitask regularisation, each 'task' choosing different coarse graphs (number of nodes, etc.) and couplings between them and other graphs in the pyramid structure. This can create an ensemble approach, where each task leads to a different realisation, since one network is trained over all outcomes. A multi-scale coherence step from previous work is also extended to become feedforward, avoiding dense matrix inversions.

Critical strengths

As far as I am aware, there has not been any work in creating an ensemble/multitask regularisation, and the contributions, as a consequence, appear to be novel and interesting. The experimental section consistently shows that performance increases with 'number of training hierarchies'. It appears that sampling hierarchies leads to informative representations, perhaps by reinforcing key motifs consistent across samples. It is interesting to consider distribution-based GNN/multi-scale GNNs as potential follow ups or by products of this work. The recursive/feed-forward operator also appears to largely reduce the computational time. Further, the flow and narrative of the work is mostly clear, though the notation may be dense at some points. The experimental sections also appear to answer most questions surrounding the core principle behind the work, assessing different degrees of randomness in hierarchy distribution, etc, while also providing direct comparisons with other works in Table 1. Overall, the quality of the work appears to be high.

Critical weaknesses

There are some minor areas where the paper could be clearer. The notation is rather dense at many points, Equation contains variables and functions with three indices, while Equation 5 contains variables with four indices. Additionally, Equation 8 makes the assignment matrix S look like a function. It could made be clearer in what order the coherence/averaging over hierarchies occur.

Questions for authors

In Figure 3/4, why does Random often perform equally or better than SHR when varying number of hierarchies/randomisation level? Could it imply that the hierarchy distribution in Section 3.2 is not effective at exploring the space of connections? I think the paper could benefit from additional discussions, or maybe experiments, on this topic, as to my understanding at least one of the purposes of hierarchical graphs is to capture interactions between measured variables and their latent influences. A random approach could introduce spurious influences, somewhat going against the purpose of GNNs in these settings.

2 comments

We thank the reviewer for finding our contributions novel and interesting, and for considering the work and the experimental analysis well done.

Weakness: the notation is rather dense [...] and it could made be clearer in what order the coherence/averaging over hierarchies occur

Given the articulated data/structures we are dealing with, simplifying the notation while still keeping the formalism and readability is challenging. In order to address the request, we dropped the forecasting-horizon indices in Eq. 5 and 6, and the time index t in Eq. 7, and the parentheses after S in Eq. 8 (so that now it reads as a matrix product rather than a function). We also made the order of operations explicit in Sec. 3.1 and in the new Fig.1, which shows training and inference pipelines. Coherence is enforced within each sampled hierarchy, before averaging.

Question: In Figure 3/4, why does Random often perform equally or better than SHR when varying number of hierarchies/randomisation level? Could it imply that the hierarchy distribution in Section 3.2 is not effective at exploring the space of connections? [...]

Results in figures Fig. 3 and 4 all use variants of the SHR training on random hierarchies, and no model in these figures uses the k-MIS distribution of Sec. 3.2. For ‘Rand.’, we generate a new hierarchy for each training sample. For the other simulations in the figures, hierarchies are sampled ahead of training and are then used as the set from which to sample when training the model. This is to isolate the impact of increasing the diversity of the hierarchies used for training. So the figures essentially show that in most cases the more the diversity, the better, which is consistent with our claims.

For what concerns comparing ‘Rand.’ with other approaches to generate hierarchies (k-MIS), the direct comparison is in Tab. 2 (SHR (rand.) vs. SHR). Here, k-MIS and the ‘Rand.’ approaches indeed perform comparably, with k-MIS being clearly better on EngRAD. As you suggest, the increased diversity brought by randomization is likely why performance is similar to the more principled approach in several settings. Moreover, randomizing the hierarchy does not discard the structural information, which is preserved in the input graph at the bottom level of the hierarchy. We agree that this point deserves more discussion, and we have expanded it in App. C.1 (paragraph ‘Comparison of hierarchy distributions‘) of the revised version, and we will also include a comment on this in the main paper (Sec. 5). Finally, we will improve the Figures' description.

Thanks for your response and for adding discussions

Comments on the paper

1 comment
AuthorsOpenReview ↗

Updates in the revised version of the paper

We thank all reviewers for their feedback. We have updated the paper with all the changes described in our answers. In particular:

  • We added Tab. 8, which reports the bottom-level performance of SHR on single k-MIS hierarchies without inference-time averaging and we moved the results without averaging to the main body (Fig. 5).
  • We added Tab. 9, which reports the performance of SHR at all hierarchy levels on an in-distribution k-MIS hierarchy and an out-of-distribution spectral hierarchy, compared with ‘Fixed hier.’ trained and tested on each hierarchy.
  • We extended the inductive experiments with an additional setting on LargeST (train on GLA and GBA, test on SD).
  • We expanded the discussion on k-MIS and random hierarchy distributions (Sec. 5 and App. C.1 - paragraph 'Comparison of hierarchy distributions').
  • We improved the notation of Eqs. 5, 6, 7 and 8.
  • We updated Fig. 1 and its description to clarify the order of the steps in the pipeline.
  • We expanded the related work with the suggested references on hierarchical forecasting.
  • We also provide the code here .

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.