- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+2)
Read review
Paper summary
The paper introduces STORM, a family of gated recurrent architectures for multivariate time series using simplicial complexes. STORM embeds simplicial convolutional filters directly into the recurrent gates. This is different from prior work which usually uses simplicial complexes in the preprocessing step. This allows topology to shape hidden-state memory updates across multiple simplex orders.
Critical strengths
The idea to combine the simplicial complexes and a gated architecture is novel and interesting. The method is especially strong on long forecasting time horizons, which makes it especially interesting. It also proposes a lighter version with less parameters, while still maintaining performance.
Critical weaknesses
To my understanding, on the traffic datasets, the only measurements are at nodes (sensors). The method “create” edge and triangle signals by averaging the node values around them. So those higher-order signals aren’t actually measured, they’re just smoothed versions of node data. Because of that, when STORM/Light STORM improves over other models, I am not sure the gain comes from real higher-order topology. It might just be getting extra averaged features. For example, Water, which has edge measurement, the improvement of the methods are much smaller than with the traffic dataset. This is the main concern I have. I also did not understand why other methods in related work could not be compared against.
Questions for authors
- Why is it not possible to test the method on large-size complexes, as discussed in F.1.1?
- It seems that the claim for “LightSTORM using up to roughly two orders of magnitude fewer parameters than other graph baselines” is overstated? I don’t see this in table 10.
- As I am not well-versed in the literature, could you explain and potentially add to the appendix why you could (or not) compare against the methods mentioned in the related works?
- Finally, related to my weakness, how can you show that the method does work better because it accurately models higher-order topology? Can we use datasets with node/edge/face measurements for example?
5 comments
Author Response to Question Q4 - Part 1 of 2.
Q4: Thank you for raising this important question. We investigated it through the ablation in Appendix H.3, additional control experiments, and a new dataset with native node and edge signals.
In Table 29 (Appendix H.3), we compare graph recurrence with node–edge and node–edge–triangle recurrence on the same node-forecasting task. Adding edge states improves performance in all six evaluated settings, while adding triangle states improves it in five out of six. For example, on PeMS-D8 at S=80, the error decreases from 0.3635 with graph recurrence to 0.2218 with node–edge–triangle recurrence.
To investigate whether these improvements arise simply from averaged edge inputs, we conducted an additional ablation (Table C.1). Removing edge supervision while retaining lifted edge inputs gives similar performance. Moreover, setting observed edge inputs to zero while retaining latent edge states and their boundary-based coupling reduces node NMSE compared with graph recurrence by 36.6% on PeMS-D4 and 36.2% on PeMS-D8.
Table C.1. Node–edge control at . Final-step node NMSE (mean ± standard deviation over 3 seeds; lower is better). Bold: best. Italics: second best.
| Variant | PeMS-D4 | PeMS-D8 |
|---|---|---|
| Graph recurrence, nodes only | 0.4811 ± 0.0083 | 0.3642 ± 0.0117 |
| Node–edge, lifted edge input + node–edge loss | 0.3152 ± 0.0118 | 0.2287 ± 0.0033 |
| Node–edge, lifted edge input + node-only loss | 0.3061 ± 0.0081 | 0.2146 ± 0.0123 |
| Node–edge, latent edge state + node-only loss | 0.3049 ± 0.0154 | 0.2324 ± 0.0085 |
We also evaluated three node forecasters (DCRNN, Graph WaveNet, and STID) followed by deterministic lifting to edges and triangles, as suggested by another reviewer. As shown in Tables C.2 and C.3, the best STORM/LightSTORM variant achieves lower pooled multi-order NMSE than all three lifted baselines at every horizon .
Table C.2. Node forecasting followed by deterministic lifting on PeMS-D4. Final-step pooled NMSE (mean ± standard deviation over 5 seeds; lower is better). Bold: best. Italics: second best.
| Model | S=5 | S=10 | S=20 | S=40 | S=60 | S=80 |
|---|---|---|---|---|---|---|
| DCRNN + lift | 0.0483±0.0002 | 0.0789±0.0028 | 0.1930±0.0311 | 0.4018±0.0181 | 0.6548±0.0212 | 0.8595±0.0635 |
| Graph WaveNet + lift | 0.0500±0.0005 | 0.0756±0.0031 | 0.1407±0.0056 | 0.2899±0.0117 | 0.3969±0.0337 | 0.5179±0.0308 |
| STID + lift | 0.0488±0.0002 | 0.0762±0.0035 | 0.1356±0.0024 | 0.2678±0.0043 | 0.3558±0.0048 | 0.4031±0.0079 |
| Best STORM/LightSTORM | 0.0491±0.0002 | 0.0725±0.0004 | 0.1206±0.0020 | 0.2247±0.0033 | 0.2686±0.0031 | 0.3164±0.0017 |
Table C.3. Node forecasting followed by deterministic lifting on PeMS-D8. Final-step pooled NMSE (mean ± standard deviation over 5 seeds; lower is better). Bold: best. Italics: second best.
| Model | S=5 | S=10 | S=20 | S=40 | S=60 | S=80 |
|---|---|---|---|---|---|---|
| DCRNN + lift | 0.0408±0.0006 | 0.0619±0.0031 | 0.1326±0.0037 | 0.3173±0.0332 | 0.5309±0.0380 | 0.7433±0.0520 |
| Graph WaveNet + lift | 0.0411±0.0018 | 0.0566±0.0015 | 0.1149±0.0057 | 0.2357±0.0088 | 0.3497±0.0255 | 0.4582±0.0729 |
| STID + lift | 0.0375±0.0004 | 0.0545±0.0008 | 0.1009±0.0023 | 0.2057±0.0027 | 0.3116±0.0046 | 0.3838±0.0065 |
| Best STORM/LightSTORM | 0.0402±0.0005 | 0.0541±0.0007 | 0.0953±0.0019 | 0.1592±0.0093 | 0.1951±0.0105 | 0.2310±0.0059 |
Together, these results suggest that the improvements on PeMS are not simply due to averaged features, but also reflect the benefit of coupled higher-order recurrent states.
Author Response to Question Q4 - Part 2 of 2.
To complement these experiments, we evaluated STORM on the public PSML power-system dataset, which has a fixed topology and provides native voltage angles on 23 buses and active-power flows on 30 transmission lines. The boundary operator is the physical bus–line incidence matrix. We used 60,000 consecutive one-minute samples, aggregated into 12,000 five-minute samples, and evaluated horizons S=20,60,80 with three seeds. As shown in Table C.4, the best STORM/LightSTORM variant achieves the lowest final-step pooled node–edge NMSE at all three horizons, reducing error relative to the strongest competing baseline by 37.4%, 27.2%, and 19.9%, respectively.
Table C.4. PSML forecasting results. Final-step pooled node–edge NMSE (mean ± standard deviation over 3 seeds; lower is better). Bold: best. Italics: second best.
| Model | S=20 | S=60 | S=80 |
|---|---|---|---|
| LSTM | 0.006768 ± 0.000628 | 0.027643 ± 0.000620 | 0.024004 ± 0.000583 |
| xLSTM | 0.008226 ± 0.000080 | 0.031128 ± 0.000990 | 0.029276 ± 0.004377 |
| Transformer | 0.008455 ± 0.000427 | 0.025807 ± 0.000908 | 0.020966 ± 0.000219 |
| iTransformer | 0.004961 ± 0.000339 | 0.024889 ± 0.000060 | 0.025914 ± 0.001394 |
| GCN | 0.051488 ± 0.009440 | 0.067024 ± 0.001887 | 0.074483 ± 0.014218 |
| GCN-xLSTM | 0.007053 ± 0.001021 | 0.020820 ± 0.001018 | 0.018586 ± 0.001126 |
| SC-VAR | 0.014630 ± 0.000196 | 0.051733 ± 0.000156 | 0.053974 ± 0.000197 |
| Best STORM | 0.003108 ± 0.000054 | 0.015149 ± 0.001027 | 0.016218 ± 0.002434 |
| Best LightSTORM | 0.004063 ± 0.000250 | 0.015687 ± 0.001625 | 0.014883 ± 0.002360 |
Finally, we agree that evaluating STORM on datasets with native node, edge, and face measurements would be valuable. However, finding established temporal forecasting benchmarks with native signals across all three simplex orders remains challenging, as discussed in Appendix I.
We hope our responses and additional experimental results have addressed the reviewer's questions and concerns. We remain happy to provide further clarification if needed.
Author Response to Questions Q1-Q3
We thank the reviewer for taking the time to review our work and for appreciating the novelty of the architectural and the empirical results. We appreciate the questions and concerns raised and address them individually below, hoping to clarify the main points.
Q1: Testing STORM on larger complexes is possible. In Appendix F.1.1, we simply state that the datasets used in our experiments contain between 76 and 676 simplices. These are the sizes of the selected benchmarks, not a limitation imposed by the model. As discussed in Appendix D, for a fixed filter degree and bounded simplex degree, the computational cost scales linearly with the number of simplices. We apologize for any confusion and will clarify this in the revised manuscript.
Q2: Thank you for pointing this out. The statement appears in the conclusion, where we discuss the node-only forecasting experiments. We were referring to the results and parameter counts reported in Table 4, rather than Table 10, which reports the parameter counts for the joint multi-order experiments. For example, on PeMS-D4, STAEformer has 1.59–2.07 million parameters, while LightSTORM has 33.9–65.9 thousand parameters. This is the comparison behind our statement that LightSTORM uses up to roughly two orders of magnitude fewer parameters. We will make this distinction clear and add an explicit reference to Table 4 in the conclusion.
Q3: We selected the baselines to compare different ways of modeling spatial and temporal information. LSTM, xLSTM, Transformer, and iTransformer provide comparisons without topological operators. GCN and GCN-xLSTM use graph-based spatial information, while SC-VAR and SCRNN represent existing approaches to temporal modeling on simplicial complexes. In particular, SCRNN applies simplicial convolutions before recurrence, whereas STORM includes them directly inside the recurrent gates. We also compare with DCRNN, Graph WaveNet, STID, and STAEformer in the node-only forecasting task (Table 4). Regarding Chen et al. (2022b, 2023), both methods use higher-order topological information but focus on predicting node-level signals. In contrast, our main task jointly predicts signals across all simplex orders. Therefore, these methods cannot be directly evaluated on the main multi-order task without adapting their architectures. We discuss our baseline selection in Appendix F.2.1 and will add a brief explanation of why some related-work methods are not directly comparable to our main multi-order forecasting task.