acceptodds
Withdrawn from ICLR 2027

Priors over Systems: Synthetic Pretraining for Spatio-Temporal Forecasting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Spatio-temporal foundation models aim to transfer across networks, but current pretraining relies on a limited collection of real systems. We ask whether a sufficiently expressive generative prior can replace real-system data collection with synthetic pretraining. MicroFlow generates synthetic systems in three separated stages: it draws a latent constitution that fixes how the system may behave, simulates its evolution from micro-level interactions, and then adds an observer that aggregates the granular trajectories into time series. We find that this separation of dynamics from observation makes the prior more expressive than naive functional series propagation. To demonstrate the usefulness of MicroFlow, we develop MicroFlow-ST, an architecture that models each modality (time series and graphs) with its own tower and couples them for spatio-temporal training. Without observing any real system during pretraining, our frozen checkpoints are competitive with similarly-sized state-of-the-art foundation models trained on both synthetic and real-world data, outperforming all of the ones we evaluate on zero-shot long-term spatio-temporal forecasting. Lightweight adaptation of MicroFlow-ST further lowers its MAE by 33% and 42% on average across short- and long-horizon tasks, respectively. We release the synthetic generator and all reported checkpoints.

Primary area:
learning on time series and dynamical systems
Keywords:
spatio-temporal forecasting, prior-data fitted networks, synthetic pretraining, foundation models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 5 reviews, above 27% of the others.
Reviewer_5nmuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MicroFlow, a synthetic generative prior for spatio-temporal forecasting that separates system constitution, micro-level interactions, and observation processes to generate diverse synthetic systems. Based on this prior, the authors develop MicroFlow-ST, a dual-tower architecture that integrates temporal and structural modeling through sparse cross-modal fusion and staged pretraining. Experiments on six real-world traffic datasets demonstrate competitive zero-shot forecasting performance, particularly on long-horizon traffic-flow tasks, along with further improvements through lightweight adaptation. However, the independent contribution of graph structure and the effectiveness of the flexible observation mechanism remain insufficiently validated, while the evaluation of cross-domain generalization is limited.

Critical strengths

This paper addresses the limited diversity of real-world systems available for pretraining spatio-temporal foundation models, presenting a clear and practically meaningful research motivation. The authors propose MicroFlow, a synthetic generative prior that separates system constitution, micro-level interactions, and observation processes to generate systems with diverse topologies, dynamics, and measurement mechanisms. They further develop MicroFlow-ST, which integrates temporal and structural modeling through a dual-tower architecture. The proposed framework is relatively comprehensive, with detailed methodological descriptions and implementation settings. The experiments include zero-shot forecasting, lightweight adaptation, and several controlled comparisons, demonstrating the potential of purely synthetic pretraining for transfer to real-world systems, particularly with competitive performance on long-horizon traffic-flow forecasting tasks.

Critical weaknesses
  1. Limited validation of the contribution of graph structure. The full MicroFlow-ST model underperforms the standalone Temporal Tower on all six short-term zero-shot forecasting tasks. The paper lacks controlled ablation studies on graph structure in the main forecasting experiments, making it difficult to assess the actual contribution of graph information to prediction performance.

  2. Insufficient evaluation of the flexible observation mechanism. MicroFlow supports one-to-many and many-to-many associations between system entities and observed channels. However, the real-world forecasting experiments do not evaluate such complex measurement associations, leaving the practical benefits of this design insufficiently validated.

  3. Limited evaluation of cross-domain generalization. The experiments mainly focus on traffic datasets, with limited evaluation on other application domains. Additional spatio-temporal forecasting experiments on more diverse systems are needed to demonstrate the generalization ability of the proposed framework.

Questions for authors
  1. Compared with existing synthetic-only pretraining methods such as NEO ST, what new modeling capabilities does MicroFlow provide through its Constitution–Interaction–Observation framework?

  2. Why does the complete MicroFlow-ST (Stage 2) underperform the standalone Temporal Tower (Stage 1a) on all six short-term zero-shot forecasting tasks?

  3. Can the authors compare forecasting performance using the original and perturbed graph structures to further demonstrate the contribution of graph information in MicroFlow-ST?

  4. Why do the GraphLand experiments use only the Stage 1b structural model rather than the complete MicroFlow-ST?

  5. Can the authors provide experiments with non-one-to-one associations between system entities and observed channels to validate the effectiveness of the proposed flexible observation mechanism?

  6. Why does Table 1 directly average raw MAE values across speed and flow datasets with different physical units to compare overall model performance?

Reviewer_z4ZkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses a narrow data source in spatio-temporal foundation models: public corpora contain far fewer distinct real systems than available time windows, and repeatedly windowing the same networks brings no new generation mechanisms, interactions, or measurement schemes. The authors therefore generate systems instead of collecting them, and introduce MICROFLOW, a three stage generator that samples a system constitution, simulates its evolution through micro level interactions, and then uses a separate observer layer to aggregate the resulting trajectories into multivariate time series, so that dynamics and observation are separated and the prior can express cyclic causality, several channels per entity, and a graph that constrains observations without being recoverable from them. Built on this prior, MICROFLOW-ST trains a temporal tower and a structural tower separately and couples them through bridges routed by the observed graph and the measurement relation, using a staged recipe with zero initialized gates so that composition does not disturb what either tower has already learned. A single frozen checkpoint that has never seen a real system remains competitive in zero shot transfer to six real traffic networks and outperforms every model evaluated on long horizon forecasting, and lightweight adaptation on a small slice of target data lowers mean absolute error by 33 percent and 42 percent on short and long horizon tasks respectively, although the authors note that the prior favors systems of the physical transport type and loses to a purely functional prior on speed datasets.

Critical strengths
  1. The authors argue that the prior is a mechanism that can be tested rather than a batch of synthetic data. They decompose it into three layers: first a system constitution, that is, its topology and parameters; then particles interacting on that structure under micro level rules, which advances the system state step by step; and finally a separate observer layer that aggregates the fine grained trajectories into time series. The value of this decomposition is that it lets the authors state, before any experiment, which three properties it is meant to guarantee: a quantity can influence its own future, one entity can carry several measurement channels, and the graph constrains the observations without being recoverable from them. More importantly, the controlled experiment in Section 6.4 fixes the architecture, the training budget, and the evaluation protocol, and changes only the prior, from the mechanistic one to a purely functional one. This is the only experiment in the paper that isolates the variable claimed in the title. The outcome is again not one sided: all eight flow cells favour the mechanistic prior, while all four speed cells favour the functional prior. A structured split of this kind is hard to obtain by tuning, and it also bounds the claim to systems of the transport type, which shows that the authors know where their boundary lies.
  2. The staged composition in the paper has both a clear justification and experimental validation. The layer that merges the two backbones uses a hyperbolic tangent gate whose multiplier is zero at initialization, so the starting point of the second stage is mathematically equivalent to two independently trained backbones in parallel, and it departs from that point only as training opens the gate. This guarantees that composition itself cannot destroy the representations the two towers have already learned. The authors verify this with two sets of experiments: at an identical budget, staged initialization wins ten of twelve cells and lowers the average error by about three percent relative to training from scratch; and when adaptation of the forecasting head alone is compared with adaptation of the bridges plus the head, the paired model stays close to the temporal model alone until the bridges become trainable, and the validation error falls clearly only once the bridges are unfrozen. These two results rule out that the gains come from a larger model and more data, because the only difference between the two settings is how the modules are composed.
Critical weaknesses
  1. The core claim of the paper lacks a comparable control. The authors report no domain specific baseline trained on the target data at a matched budget, such as a strong spatio temporal graph network under the BasicTS recipe or gradient boosted trees, and the comparison runs only between foundation models, while Stage 1a with 15.8M parameters already obtains the lowest average MAE in the whole table on the short horizon in Table 1, which shows that a single dataset reference is missing and leaves the statement that zero shot performance is close to optimal without a basis for comparison. The three seeds reported in Tables 18 and 19 are inference seeds, and they measure how stable the evaluation is when the weights are held fixed. The claims made in those two tables, however, compare models that were trained independently, and the paper reports no training seed variance anywhere.
  2. None of the three properties the paper establishes is actually triggered in the evaluation. Section 4.1 takes cyclic causality, multivariate observability, and latent topology as the basis for the prior design, yet in the six traffic datasets each sensor corresponds to exactly one channel and in GraphLand each attribute column is a per node field, so the association relation is an identity mapping throughout and K equals V; the case where one channel aggregates several entities, which is the key case for this section, never appears, and no experiment tests the property that the graph constrains observations without being recoverable from them. At the level of the prior there is only one comparison, the full mechanistic prior against a functional prior, and the three properties are never ablated separately, so even if Section 6.4 supports the claim that mechanism beats function, it cannot show which property is responsible. The novelty of the paper rests precisely on these three properties, and since they are neither triggered nor decomposed in the experiments, the core design claim remains at the level of a design statement.
Questions for authors
  1. Tables 18 and 19 report means over inference seeds only. Could the authors retrain Stage 2 with at least three training seeds while reusing the Stage 1 backbones, and then run a paired test across the twelve dataset and horizon cells?
  2. Section 3 allows the number of measurement channels to differ from the number of entities, yet in the six traffic datasets each sensor corresponds to exactly one channel and in GraphLand each attribute column belongs to its own node, so the association relation is an identity mapping throughout the evaluation. Could the authors state this explicitly in the text and add a synthetic setting in which one channel aggregates several entities?
Reviewer_DLirOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes MicroFlow, an approach following Prior-Data Fitted Networks to deal with cross-modal modeling problem between graphs and time series. Firstly, the authors introduce a synthetic data generation method that uses particle-based simulation to generate a large number of graph–time-series pairs, which aims to provide an expressive prior for model pretraining. Second, the authors propose MicroFlow-ST, a model for spatio-temporal forecasting pretrained on data generated by MicroFlow. MicroFlow-ST uses a dual-tower architecture to model graphs and time series separately, and further trains the model with cross-modal, graph-conditioned bridges to learn their coupling. Experiments on six traffic datasets show that MicroFlow has strong zero-shot forecasting performance, with further improvements through lightweight fine-tuning.

Critical strengths

Large-scale synthetic data generation. The paper provides a well-designed pipeline for generating paired graph and time-series data, including graph topology construction, particle-based dynamics, exogenous drivers, and observation mechanisms, enabling the generation of a large and diverse dataset of graph–time-series pairs.

Competitive performance. The model achieves competitive zero-shot forecasting performance on six real-world traffic datasets using only synthetic data for pretraining on long-term forecasting tasks. Lightweight fine-tuning further improves its performance.

Critical weaknesses

Limited benefit from cross-modal modeling. The proposed dual-tower architecture does not seem to bring consistent improvements. Stage 2 performs worse than Stage 1a on all six short-term forecasting datasets, however, using a larger model with an additional graph tower. The experiments do not clearly demonstrate that cross-modal modeling provides additional benefits over the temporal-only model. A better training framework may be needed to make more effective use of cross-modal information.

Unclear contribution of system diversity. The paper argues that system diversity is more important than the number of temporal windows for pretraining. However, it remains unclear whether the performance gains come from an expressive system prior, or just come from simply more synthetic data. An ablation comparing different numbers of systems with the same total number of training windows would help clarify this.

Questions for authors
  1. The limited gains from cross-modal modeling may be due to the limited expressive capacity of the architecture and training strategy. Do the authors think a unified architecture with joint attention, such as MMDiT, could be more effective for this task?

  2. Both NeoST and MicroFlow generate time series from latent dynamics on graphs. What are the key limitations of NeoST's generative prior that MicroFlow aims to address?

Reviewer_SZRGOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studied whether models pretrained entirely on synthetic data can generalize to real-world spatio-temporal forecasting. The authors propose MicroFlow, a synthetic system generator, and MicroFlow-ST, a model with temporal and structural backbones. MicroFlow creates synthetic networks and simulates their behavior to generate corresponding time-series data. MicroFlow-ST first trains its temporal and graph components separately, then combines and trains them together for forecasting. Experiments on six traffic datasets show strong forecasting performance.

Critical strengths
  1. This paper identified an important limitation of existing methods and provided a reasonable method to synthesize pretraining data with meaningful structural and temporal dependencies.

  2. The experiments demonstrate the good performance of the proposed methods.

  3. The proposed methods exhibit a certain degree of reusability. The temporal backbone, structural backbone, and composed model can be used separately for time-series forecasting, node-property prediction, and graph-conditioned forecasting.

Critical weaknesses
  1. Insufficient ablation of the prior’s individual mechanisms. The paper highlights several key components, including micro-level simulation, dynamic feedback, and the separation of dynamics from observation. However, there is no comprehensive ablation analysis of each component. Consequently, it remains unclear which specific mechanisms contributed to the observed performance improvements.

  2. Several experimental results warrant more cautious interpretation. For example, in the comparison between staged training and training from scratch, only the training budget for Stage 2 is taken into account. However, in reality, models trained via a staged approach also benefit from additional, separate phases of temporal and structural pre-training. This comparison therefore does not fully distinguish the benefit of the staged training strategy from that of additional pretraining. Besides, some baseline results are also taken from published papers rather than reproduced within a common evaluation pipeline.

Questions for authors
  1. Please provide targeted ablations to isolate the contributions of micro-level simulation, dynamic feedback, and the separation of dynamics from observation. In particular, which mechanisms drive the observed improvements most, and does each provide a measurable benefit when the architecture and training budget are held constant?

  2. Does the advantage of staged training over training from scratch persist when the total training budget is matched, including the additional temporal and structural pretraining in Stages 1a and 1b?

  3. For baseline results taken from published papers, could the authors clarify how consistency in preprocessing, data splits, forecasting horizons, and evaluation metrics was verified?

Reviewer_JEVpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a modular dual-tower foundation model architecture, dubbed MicroFlow for handling spatio-temporal data. The pre-training data is completely synthetic and is based on simulated real-world physics and can be deployed for real-world networks with zero-shot forecasting.

Overall: The paper presents a clear architecture for handling spatio-temporal data using foundation models but its claims are not rigorously tested for real-world applications.

Critical strengths

(a) The proposed model architecture is completely trained on synthetic data and either matches or outperforms state-of-the-art foundation models based on the experiments shown in the paper. (b) The learned model is shown to have excellent zero-shot capabilities on real-world graph topologies at inference even though it was trained on synthetic graphs with missing or spurious edges.

Critical weaknesses

(i) The experiments are limited to traffic settings and do not include real-world systems which are highly volatile such as financial limit-order books or macro-liquidity flows. (ii) The wins seem to come from long-term forcecasting which maybe an issue since they naturally have mean-reverting or cyclical trends. (iii) The paper does account for graph corruptions at inference but it does require an explicity graph structure to be passed into the model and does not infer it from the provided time-series data.

Questions for authors

(1) How can this be generalized to other spatio-temporal systems such as financial transaction graphs which are not physics based? (2) How does the model perform for shorter forecasting horizons which might have more chaotic fluctuations?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.