acceptodds
Under review as a conference paper at ICLR 2027

Learning Spatio-Temporal Foundation Models for Traffic Prediction from Pure Synthetic Data

Abstract

Spatio-temporal foundation models (STFMs) offer a promising path toward universal traffic forecasting by learning transferable priors across heterogeneous road networks and cities. However, existing approaches suffer from distributional bias in real-world pre-training data, structural bottlenecks of autoregressive or diffusion-based paradigms, and objectives that overemphasize point-wise reconstruction in noisy observation space. We propose NeoST, the first STFM for traffic forecasting pre-trained solely on procedurally generated synthetic systems. NeoST introduces a scalable synthetic pre-training corpus to mitigate real-world bias, a latent-space reasoning architecture that generates and iteratively refines multiple future trajectories without sequential error accumulation, and latent-space objectives that emphasize structural dynamics and enable inference-time correction under distribution shifts. Extensive experiments across diverse real-world traffic benchmarks show that NeoST consistently outperforms existing STFMs in diverse real-world traffic systems, achieves superior long-horizon stability and inference efficiency.

Primary area:
learning on time series and dynamical systems
Keywords:
Spatio-Temporal Data Mining, Traffic Forecasting, Foundation Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XiksOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a spatio-temporal traffic foundation model named NeoST, which is pre-trained solely using synthetic data. The synthetic data engine simulates second-order stochastic latent dynamics on random graphs through Chebyshev filtering, re-samples generator parameters for each sample, and generates observations through random projection and masking methods. The encoder alternates between node-level temporal attention and node-to-node spatial attention. The reasoner generates multi-scale latent trajectories using multi-stride ChebConv-LSTMs, fuses them through dynamic convolution, and refines them with a Transformer. In the pre-training stage, the output after PCA projection is aligned with the generator’s latent states, plus an observation reconstruction term. In the inference stage, a ridge regression output layer is fitted on the encoder features.

Critical strengths

The pre-training stage uses procedurally generated, graph-coupled stochastic systems, with the generator’s future latent states as supervision information, which is not available in real data. The generator’s Markov structure, heteroscedasticity, spectral bounds, and moment bounds are formally described in Appendix B. This provides a controllable and scalable method to alleviate the data scarcity and bias in spatio-temporal foundation models.

Critical weaknesses

W1. According to Appendix C.4, the ridge regression readout is fitted based on the observations of the target dataset itself, while baselines such as TimesFM and FactoST receive no target-domain calibration. Moreover, it is unclear whether this fitting uses a single window or the entire dataset, and whether it includes the test period. Under LODO, the 96-step results at 137.25 for PEMS03, 221.97 for PEMS08, and 15.08 for PEMS-BAY are all worse than all the baselines in Table 1. Therefore, the long-horizon advantage mainly comes from in-domain calibration, which leaves the core claim that synthetic pre-training brings cross-domain generalization without support.

W2. In Equation 4, the PCA axes are arranged in the order of variance, and the signs are arbitrary, while the basis of HH comes from the random weights of each sample. Since x=Hwx = Hw identifies HH only up to an invertible transformation, the element-wise Frobenius distance is not identifiable. HH has LNLN rows per time step, but Z^\hat{Z} has only PLNP_L N rows per patch, and there is no alignment relationship stated between them. The ChebConv-LSTM in the reasoner also rolls out step by step from the historical mean, which is still autoregressive. Therefore, the claimed latent objective and non-autoregressive inference mechanism are inconsistent with the implementation.

Questions for authors

Q1. Is the ridge regression readout based on historical data from each input window or on aggregated observations from the entire target dataset? If aggregated, in which time period is it conducted, and does it include the test period? Can the results be reported based on fitting to the input window alone, with the same linear calibration applied to the baselines?

Q2. In Equation 4, how are the per-time-step HH and the per-patch Z^\hat{Z} aligned? Is PCA recalculated per sample and back-propagated? How do you handle the ambiguity in the order and signs of each component, and the arbitrary basis of HH?

S1. Please explain why TimesFM has the same results for METR-LA and PEMS-BAY data in Table 1.

S2. Pre-training NeoST on real data such as LargeST, or on a mixture of real and synthetic data, under the same compute budget and comparing the zero-shot results, would directly test the contribution of synthetic pre-training.

Reviewer_6jigOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes NeoST, a spatio-temporal foundation model for traffic forecasting pretrained entirely on synthetic data. The authors introduce a stochastic graph-based data generator to produce diverse spatio-temporal dynamics, a hierarchical latent-space reasoning architecture for multi-scale forecasting and trajectory refinement, and a latent alignment objective combined with test-time adaptation. Experiments on six real-world traffic datasets demonstrate competitive zero-shot forecasting performance, particularly for long-term prediction.

Critical strengths
  1. The paper addresses an important problem in spatio-temporal forecasting, particularly the scarcity and limited diversity of real-world training data. Exploring synthetic data for foundation model pretraining is a meaningful and promising direction.
  2. The paper is well-structured and easy to follow. The motivation, methodology, and experimental evaluation are presented clearly, making the overall framework straightforward to understand.
Critical weaknesses
  1. The proposed synthetic data is generated from randomized graph dynamical systems, but it remains unclear whether these systems capture meaningful real-world spatio-temporal patterns rather than arbitrary random dynamics. Although the authors introduce various dynamic mechanisms to increase diversity, diversity alone does not guarantee realistic or transferable representations. The t-SNE visualization provides limited evidence, particularly regarding long-term temporal dependencies and spatial interactions, which are critical for traffic forecasting. More comprehensive quantitative evaluations of synthetic-to-real distribution similarity are needed. Comparisons with existing data-driven generation approaches, such as GAN-based methods, would also help justify the effectiveness of the proposed synthetic generator.

  2. The paper emphasizes zero-shot generalization, yet the main experiments explicitly employ target-domain test-time adaptation (TTA). While TTA does not necessarily contradict zero-shot transfer, the distinction between frozen zero-shot inference and target-domain calibration should be clearly established. The paper also lacks sufficient details regarding the amount of target-domain data used for adaptation and whether calibration is performed independently for each test window. Although the authors provide results without TTA, the main comparisons do not adequately isolate the contribution of synthetic pretraining from target-domain adaptation. A more transparent and consistent evaluation protocol is necessary to substantiate the zero-shot claims.

  3. A major concern is whether the proposed model and the baselines are evaluated under comparable adaptation protocols. NeoST combines synthetic pretraining with target-domain TTA, whereas the baselines rely on different pretrained checkpoints or retraining procedures. Although the authors identify certain baselines with overlapping pretraining datasets, it remains unclear whether all baselines are given equivalent opportunities for target-domain adaptation. Without this clarification, the fairness of the comparison is difficult to assess. Furthermore, the reported improvements cannot be clearly attributed to synthetic pretraining, as architectural differences, model capacity, and TTA may also contribute substantially. Controlled experiments comparing real-only, synthetic-only, and mixed-data pretraining under the same architecture and consistent adaptation settings are necessary to isolate the benefits of the proposed synthetic data generation strategy.

Questions for authors

In Equation (4), the authors directly compute the MSE between PCA-projected representations and the synthetic latent states. However, PCA primarily identifies directions of maximum variance rather than explicitly aligning two representation spaces. Could the authors clarify how the PCA-projected features are aligned with the synthetic hidden states, especially when different synthetic systems have independently sampled parameters? Additionally, how is PCA implemented during training, and how are gradients propagated through this operation? Further explanation of why this alignment preserves meaningful spatio-temporal dynamics would be helpful.

Reviewer_MwuQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to pre-train a spatio-temporal foundation model using only synthetic data and transfer it to real-world traffic forecasting tasks. The authors propose NeoST, which generates synthetic data with diverse spatial structures and temporal dynamics using random graphs and stochastic graph dynamical systems, and uses the latent states from the generation process as additional supervision. NeoST adopts a hierarchical latent-space reasoning architecture that produces future latent representations through multi-scale prediction and trajectory refinement. It is trained with a latent alignment loss combined with an observation reconstruction loss. At test time, the model adapts its outputs via ridge regression. The authors conduct zero-shot forecasting experiments on six real-world traffic datasets, along with ablation studies and inference efficiency analysis. They report competitive forecasting performance across most evaluation settings.

Critical strengths
  1. A valuable research direction: pretraining purely on synthetic data. The paper explores whether a spatio-temporal foundation model can be pretrained using only synthetic data. It generates training data with diverse dynamics through random graph dynamical systems and uses the latent states available from the synthetic systems as supervision. This offers a meaningful way to reduce the model's reliance on real-world traffic data for pretraining.

  2. Evaluation on multiple real-world traffic datasets. The paper evaluates both short-term and long-term forecasting on six real-world traffic datasets and compares against a range of spatio-temporal and time-series foundation models, achieving generally competitive results. The authors also include ablations on the main model components, the training objectives, and test-time adaptation, as well as an inference efficiency comparison. Together, these experiments provide useful evidence of how the proposed method performs in practice.

Critical weaknesses
  1. The baseline results and evaluation settings in the main experiments need further clarification. Table 1 shows that NeoST achieves competitive performance on several real-world traffic forecasting datasets. However, the sources of some baseline results and the consistency of their evaluation settings remain unclear. Some reported baseline values also appear to differ from those in the original publications. It would be helpful to clarify which results were reproduced and which were taken from prior work, as well as the relevant pretraining data and evaluation protocols, better to assess the reliability and fairness of the comparisons.

  2. The ablation results do not fully support some of the claims about the hierarchical latent-space reasoning architecture. The paper evaluates Multi-Scale Prediction (MS) and Trajectory Refinement (TR) through ablation studies in Figure 4. However, the benefits of these modules appear to vary across loss weights, datasets, and prediction horizons. In some settings, removing both modules yields a lower MAE, which is not fully consistent with the claimed complementary effects and overall advantages of the full model.

Furthermore, the experiments mainly report aggregate errors at different prediction horizons. Additional analysis would help clarify how TR reduces error accumulation and how MS improves long-horizon forecasting stability.

  1. The evidence that synthetic data mitigates distributional bias in real-world data remains limited. The paper presents pretraining purely on synthetic data as one of its main contributions and emphasizes its potential to mitigate distributional bias in real-world pretraining data. While the experiments demonstrate competitive forecasting performance across multiple real-world traffic datasets, they do not directly establish how synthetic pretraining mitigates such bias. Further evidence or clarification would help support this claim and better establish the scope of the contribution.
Questions for authors
  1. On baseline results and pretraining data. Could the authors specify the pretraining data and model versions used for each baseline, and indicate which results in Table 1 are taken from prior work and which were re-evaluated? Could the authors also check for any overlap between the pretraining and test datasets, and mark the affected results accordingly?

  2. On the evaluation protocol of the main experiments. Could the authors clarify how the input lengths were handled across different baselines, how hyperparameters were selected, and what data each model was allowed to use at test time? In particular, when NeoST fits the ridge regression output adapter at test time, which observations are used, and is the adapter fitted independently for each test window?

  3. On the roles of MS and TR. The gains from the two modules in Figure 4 vary across settings. Could the authors explain these variations, clarify which loss weight was used for Table 1, and confirm whether the other experimental settings are consistent with Figure 4? Could the authors also explain how the current experiments support the claimed roles of MS and TR in long-horizon stability and controlling error accumulation?

  4. On synthetic data and distributional bias. Could the authors further explain how pretraining purely on synthetic data helps mitigate the distributional bias associated with real-world pretraining data? Would additional experimental evidence or a comparison of relevant statistical characteristics between synthetic and real-world data help support this claim?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for All Reviewers, Program Chairs, and Readers

We thank the three reviewers for their feedback. Key questions covered synthetic data realism versus real traffic and GANs, evaluation protocol fairness including baselines and test-time adaptation, MS/TR ablation roles, and latent alignment identifiability in Equation 4. We clarified the PEMS family effect with per-dataset stats, ran controlled synthetic versus real pretraining experiments, re-analyzed ablations at full budget with per-step errors, and acknowledged PCA alignment limits while showing practical gains via the alpha sweep. These points do not weaken the core claims.

Regarding this paper, we would like to share our own perspective, which we believe will contribute to the wider discussion. NeoST is the first spatio-temporal foundation model for traffic forecasting that is pretrained entirely on pure synthetic data. Prior to this work, no one had attempted to use purely synthetic data for pretraining in this context. Suitable generative mechanisms had not yet been identified, and few researchers believed that this approach would be successful. We consider this to be a valuable insight. Real-world traffic data is inherently limited in scale and biased towards particular sensor networks and cities. It is also contaminated by uncontrolled noise and missing values. By procedurally generating diverse graph dynamical systems with controllable spectral properties, nonlinearity, and noise regimes, NeoST demonstrates that transferable structural priors can be learned without the use of real data. This reduces reliance on scarce and distributionally skewed observations and opens up a new direction for foundation models in this field. We believe that this contribution alone is substantial. Additionally, we propose a latent alignment loss that differs from conventional pointwise estimation in observation space. Losses computed directly on raw values often cause models to fit local fluctuations and measurement artefacts while overlooking underlying dynamical structures, such as trends, periodicities, and multiscale evolution. By supervising the model towards the principal subspace of the synthetic latent dynamics, the objective encourages the internalisation of structural properties. This approach enables more robust transfer and inference-time correction under distribution shifts. Together with the synthetic pretraining strategy, these elements form the core of the paper. Our responses to the reviewers address the specific questions raised, and we do not believe that they undermine these contributions.

Finally, we would like to point out that, in the current AI era, papers are often examined by LLMs. This naturally results in detailed questions about many specific aspects of a submission. We find this helpful for improving quality, and we are grateful to the reviewers for helping us to strengthen the paper. However, as both authors of this work and reviewers of other ICLR submissions, we would also like to offer a human perspective on evaluating papers. When a submission contains no fundamental errors, such as fabricated data, internal contradictions, or unprofessional writing, and its core contribution remains intact, it is important to consider the inspiration the work can offer readers and the broader community. For example, NeoST demonstrates that a relatively simple generative mechanism can produce substantial spatio-temporal data, sufficient to pre-train a competitive foundation model. Readers may then wonder whether, for future spatio-temporal and even time series foundation models, real data is still necessary for pretraining. Why is this simple generative mechanism effective? Can we propose better mechanisms for synthetic data? Is it necessary to use GANs or diffusion models to synthesise and augment spatio-temporal or time series data? Similarly, regarding latent space alignment, some readers may ask whether, to avoid noise from value domain losses such as MAE or MSE, computing the loss in latent space is a better way to mitigate noise for spatio-temporal and time series data. Could such a training paradigm be used for the next generation of spatiotemporal or time series foundation models? Due to page limits, we could not explore or resolve all these questions ourselves. However, if the paper is accepted, we believe that more readers will have the opportunity to engage with these ideas, which could lead to further advances in the field.

Given our passion for this research and the broader field in which it is situated, we kindly encourage all reviewers, the Program Committee Chair, and all readers of this rebuttal to consider the paper's academic contributions from a human and judgemental perspective, in addition to taking the LLM's comments into account. We also hope that reviewers will reconsider their ratings accordingly. Once again, we would like to thank all the reviewers for their valuable feedback and for helping us to improve the quality of this paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.