- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
TRACE combines a patched temporal Transformer with Gram-based and observed-topology propagation, gated residual updates, and layer aggregation. Separate models are pre-trained within traffic, transaction, and transportation domains, then evaluated without fine-tuning on held-out graphs for forecasting or graph classification.
Critical strengths
-
The architecture offers a useful interface for featureless, static-topology, and fully dynamic graphs. Associative Gram multiplication avoids materializing a dense relation matrix, while the two branches combine observed and inferred dependencies.
-
The reported transfer results are promising across three domains with compact models, including a MiNT mean AUC of 0.746 versus 0.721 for HTGN-zs. The paper explicitly distinguishes target-contaminated baselines and the transductive nine-dataset setting from zero-shot evaluation.
Critical weaknesses
-
The TSFM baselines are recent, including FlowState (2026), but the directly relevant graph-transfer methods OpenCity [1] and UniFlow [2] are cited without experimental comparison. TSFMs receive no explicit topology and use different pre-training corpora. Compare graph-aware alternatives under matched data and inputs to establish the architectural contribution beyond combining established temporal and relational components.
-
Table 4 changes the architecture while keeping the full model’s weights frozen, so degradation measures reliance on the trained configuration rather than the potential of alternative architectures. Pre-train each variant on the same source corpora, including temporal-only and standard-attention controls. Clarify whether ‘no propagation’ also disables the layer aggregator's node-wise attention [3]; otherwise it is not an independent per-series forecaster as claimed.
-
Tables 8-9 report three separately trained domain models. These experiments establish a reusable architecture and within-domain graph transfer, but do not validate the conclusion’s single model pre-trained across heterogeneous domains. Narrow that claim or demonstrate a shared checkpoint trained on mixed-domain corpora, including genuinely varying feature dimensions.
-
Only MiNT reports seed variability; forecasting tables lack uncertainty, despite a Seoul MAE margin of only 0.0010 over Chronos-2-uni. Report repeated runs and paired intervals. The corpus-size study also changes source identities, data volume, and potentially training compute together; repeated source subsets and controlled budgets are needed to isolate a diversity/scaling benefit.
-
The Gram operator lacks node-count normalization: identical embeddings yield an all-ones matrix, so message magnitudes can grow with graph size. Test normalized kernels and size-shift robustness. Specify whether Gaussian R_t is shared across snapshots/windows and report sensitivity to random draws, since this choice can change temporal consistency.
-
Associative Gram evaluation saves storage, but Section 3.4 still includes quadratic node attention through layer aggregation [3]. Its O(n(L+P)) memory bound omits attention maps unless a memory-efficient implementation is used. Clarify that implementation and the 4096-node cap at inference, and report end-to-end runtime and peak memory across graph sizes.
[1] Li, Z., et al. OpenCity: Open Spatio-Temporal Foundation Models for Traffic Prediction. TIST, 2025.
[2] Yuan, Y., et al. UniFlow: A Foundation Model for Unified Urban Spatio-Temporal Flow Prediction. TMC, 2026.
[3] Galron, Y., Sinai, H., Maron, H., and Eliasof, M. Learning from Historical Activations in Graph Neural Networks. ICLR, 2026.
Questions for authors
-
Do the gains persist against OpenCity/UniFlow and separately pre-trained temporal-only, single-relation, and standard-attention variants? Which node-attention operations remain in the frozen ablations?
-
Was any single checkpoint trained jointly across all three domains? If not, can the conclusions explicitly distinguish architectural reuse from cross-domain pre-training and transfer?
-
How are Gaussian features reused over time, and how sensitive are results to random draws, Gram normalization, and changes in active-node count?
-
Can the authors provide forecasting uncertainty, corpus-scaling results with controlled budgets, and measured runtime/memory, including the exact inference-time node-selection policy?
4 comments
Response to W4, W5, W6, Questions and References
W4 Thank you for pointing out this detail. Following your comment, we have revised the paper to include seed level variability for the forecasting experiments, in Appendix B.1 (Tables 12 and 13). In brief the standard deviation of our performances on traffic flow forecasting and MOBINS are on average 0.08 and 0.0036, which is small compared with most margins in Tables 1 and 2. On Seoul, the margin is within one standard deviation. Regarding the corpus-size study, we kindly redirect the reviewer to the dataset-level statistics presented in Table 7. We summarize them here for better clarity. All PeMS-D{5,6,10,11,12} datasets comprise 25045 timesteps, with a minimum of 150 nodes and a maximum of 820 nodes per corpus. These are of the same scale as the PeMS0{3,4,7,8} datasets, which have a similar amount of timesteps and nodes each. The settings and model size remained the same. Since each corpus contributes a fixed number of batches per epoch (see Appendix A.3 of the revision), the number of training steps grows with the number of corpora, so this study does not separate diversity from training budget, which we acknowledge as a limitation. The only change is the set of available datasets, which we used to pre-train TRACE (which is part of the study itself). Thank you.
W5 As correctly mentioned in your review, in case of identical temporal embeddings the Gram relation is a constant matrix, and the message passed to each node is the same. Following this observation, we ran a new experiment where we compare the Gram normalizations used on traffic forecasting. We compare between our employed normalization and the division by , where is the number of nodes and is the number of features. We report the results below, in zero-shot Masked MAE$\downarrow$, following the setting of Table 1:
| Gram normalization | PeMS03 | PeMS04 | PeMS07 | PeMS08 |
|---|---|---|---|---|
| Our employed normalization (1 run) | 14.16 | 19.31 | 21.04 | 14.89 |
| Division by (3 seeds) | 14.21 0.06 | 19.23 0.01 | 20.81 0.08 | 14.86 0.15 |
The differences in performance are negligible compared to our employed normalization, with the new version yielding better performances on PeMS0{4,7,8} and slightly lower on PeMS03.
Regarding the Gaussian random walk features, we draw of Equation (1) once per network prior to pre-training and we fix it. We follow the reviewer's suggestion and test the sensitivity of the zero-shot performance on the test set when drawing a different for each network 10 times. We report the results below, in zero-shot AUC$\uparrow$, averaged over the 20 test networks:
| Draw of | AUC$\uparrow$ |
|---|---|
| Average (10 draws) | 0.747$\pm$0.001 |
The ten draws do not alter the performance, with low standard deviation. This result shows that our performance is not sensitive to the random Gaussian probe matrix, in the zero-shot setting. We thank you for this question, which we feel helped us improve an important part of our contributions.
W6 Thank you for asking about this detail. We agree with the reviewer and specify that although MHSA has computational complexity, its fused scaled dot-product attention kernels do not materialize the full attention map [2, 3], which is the implementation we considered in our method. Regarding the 4096 node cap, it was originally introduced as a conservative memory safeguard for an earlier computational formulation that explicitly materialized this matrix. It was not a requirement of the proposed propagation mechanism and does not apply to the factorized implementation used in our method (also refer to our response to Q1 of reviewer 8aJz). In Appendix B.7 (Figure 5 and Table 19) of the paper revision we have included an explicit empirical complexity analysis that supports the theoretical expectations of Section 3.4. We thank you for the question again, which helped us improve an important ambiguity in the implementation details.
Regarding Questions
We thank the reviewer for all questions, whose responses are included in the Weaknesses section above.
Q1 We kindly redirect the reviewer to our responses W1 and W2.
Q2 We kindly redirect the reviewer to our response W3.
Q3 We kindly redirect the reviewer to our response W5.
Q4 We kindly redirect the reviewer to our responses W4 and W6.
In general, we have revised the paper to include experimental evidence and clarification to all issues raised by the reviewer. We thank you once again, and hope that our clarifications and additional experiments will lead to a reconsideration of your score.
References:
[1] Li et al. Open spatio-temporal foundation models for traffic prediction. ACM TIST, 2025.
[2] Rabe and Staats. Self-attention does not need memory. arXiv, 2022.
[3] Dao et al. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. NeurIPS, 2022.
Thank you for the thorough response and clarifications. The authors have addressed my previous review comments and provided detailed explanations of many aspects of the implementation. These clarifications have strengthened my understanding of this work and helped me better appreciate its novelty. Based on this, I will raise my scores.
Dear Reviewer GGcK,
We are happy to read that you found our responses to address your comments, and that they helped you to better appreciate the novelty of our work. Thank you for raising your score.
We feel that your insightful and constructive feedback helped us to improve the quality of our paper --- thank you.
With kindest regards,
Authors.
Response to W1, W2 and W3
We thank you for your careful and detailed evaluation. We are encouraged by your recognition that The architecture offers a useful interface for featureless, static-topology, and fully dynamic graphs and that The reported transfer results are promising across three domains with compact models. We answer each point below.
In short: we include (i) OpenCity-Plus [1] on all four PeMS targets, (ii) frozen and retrained ablations that remove the node attention, (iii) three seeds for the MOBINS targets and for all four PeMS targets, (iv) the measured scaling of runtime and memory, and (v) a precise statement of the scope of the unification claim. Overall, during the rebuttal we ran 9 new experiments and 3 new analyses, all listed in our general comment. We hope that you find our responses satisfactory, and that you will consider revising your score.
Regarding Weaknesses
We kindly thank you for the feedback, which we feel helped us improve and clarify very relevant and important details of our paper.
W1 Thank you for this comment. We agree with the reviewer that including a graph-based baseline would position our contribution better. Accordingly, we included OpenCity-Plus [1], a graph-based spatiotemporal foundation model, to which we compare our performance. We report in the table below the MaskedMAE$\downarrow$ of OpenCity-Plus and of TRACE zero-shot averaged over three seeds († means that the pre-training corpus of the model contains the dataset corresponding to the column, or traffic data from the same district). We have updated Table 1 of the revision to include OpenCity-Plus, and the three seed performances of TRACE in Appendix B.1.
The results show that TRACE zero-shot outperforms OpenCity-Plus on all four PeMS targets, even when the pre-training corpus of OpenCity-Plus contains the target dataset. Regarding UniFlow, the authors of the paper did not publicly release the pre-trained weights of their method, so this is not included as an additional baseline. Thank you.
W2 Thank you for this observation. We clarify the ablation definitions: no propagation removes both message passing methods considered in the paper, and with them also the node level attention, so this variant is graph-free; topology only, Gram only and Gram from X keep the attention across all nodes. We therefore extend Table 4 to remove node attention and report the results below:
When pre-training on traffic flow forecasting and leaving out PeMS08 the performances are 14.59 without node attention, 14.61 without propagation and 14.66 with the Gram relation only, against 14.89 for the full model. Thus, with this ablation we show that those model components are not driving the gains for this pre-training. Thank you, we have updated the revision with these results and clarified the ablation details.
W3 Thank you for this observation. We agree with you that in the paper the evaluation of TRACE is in-domain and therefore does not establish cross-domain generalization. Our intention was to claim that TRACE is a single model definition that can be trained on any temporal graph domain, and that it can be pre-trained on multiple graphs within each domain. Still, we follow the reviewer’s comment and propose a new experiment where we jointly pre-train TRACE on traffic forecasting and transportation networks. In particular, we follow the settings we used for Table 1 and 2, and leave-out from pre-training PeMS04 and Daegu. We compare, for a single seed, the jointly pre-trained model with the single domain model MaskedMAE$\downarrow$ in the table below:
The jointly pre-trained model is on par with, and slightly better than, the single-domain model. We have revised the paper to include this new experiment in Table 6 (Section 4.2, setting in Appendix B.4) and reframed our claims accordingly. Thank you.