acceptodds
Under review as a conference paper at ICLR 2027

Seq-Flow: Efficient Probabilistic Forecasting with Self-Rollout Error Control

Abstract

Many scientific forecasting tasks require updating a distribution over future trajectories as new observations arrive. Conventional diffusion and flow models generate each forecast from Gaussian noise, often at the cost of many sampling steps. Warm-start methods reuse earlier predictions to reduce this cost, but their models are not trained to perform the forecast update itself, which can compromise quality under few-step sampling. In this work, we introduce Seq-Flow, a conditional flow model whose ODE transports samples from the previous forecast distribution to the updated one. Because successive forecasts often differ only modestly, this transport starts from an informative distribution and can produce accurate updates with few flow evaluations. Recursive reuse also creates a challenge: errors in one forecast become errors in the initial states of subsequent flows. We address this with self-rollout training, in which a moving average copy of the model generates forecasts that initialize later training updates. Unlike self-forcing methods, which reuse generated outputs as conditioning context, Seq-Flow reuses them as the source of the next flow. Experiments On particle-accelerator beam spill forecasting show Seq-Flow reduces CRPS by 65% under a few-NFE sampling budget, while remaining competitive with strong baselines on fluid-dynamics forecasting tasks. Although trained on self-rollouts of at most four updates, Seq-Flow remains accurate over more than 400 consecutive updates.

Primary area:
generative models
Keywords:
Flow matching, dynamic systems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Fh5jOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Seq-Flow, a flow-matching framework that updates the previous forecast distribution to the new one using newly observed states, enabling efficient few-step probabilistic forecasting. It further introduces self-rollout training with EMA and renoising to mitigate error accumulation during recursive forecasting, and demonstrates strong performance on several scientific forecasting tasks

Critical strengths

The key strength is the effective forecast-to-forecast flow formulation, which reuses the previous predictive distribution instead of restarting from Gaussian noise. This provides a simple and practically meaningful way to improve few-step forecasting efficiency, with strong empirical gains across the evaluated tasks.

Critical weaknesses

The contribution of the proposed components is not sufficiently isolated. In particular, the roles of self-rollout, renoising, EMA, and the mixed training objective are not fully disentangled, making it difficult to determine how much of the performance improvement comes from the core Seq-Flow formulation itself.

Questions for authors

The paper uses a mixed training objective combining Gaussian-source and self-rollout training, but the contribution of each component is unclear. Can the authors provide an ablation comparing the two objectives separately and explain how much of the performance gain comes specifically from self-rollout Seq-Flow?

The paper claims that Seq-Flow remains stable for more than 400 recursive updates despite training with at most four self-rollout steps. How sensitive is this long-horizon stability to the choice of rollout horizon T0T_0 and renoising level τr\tau_r?

Reviewer_ZJTYOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Seq-Flow, a flow-matching-based framework for efficient online probabilistic forecasting. Unlike conventional diffusion and flow models that repeatedly generate future trajectories from Gaussian noise, Seq-Flow directly models the transition between consecutive predictive distributions by using samples from the previous forecast as the source distribution of the next flow. To mitigate error accumulation caused by recursively reusing model-generated forecasts, the authors introduce self-rollout training, in which an exponential moving average (EMA) copy of the model generates forecasts that subsequently serve as sources for training. The proposed method also incorporates partial renoising to improve robustness against accumulated prediction errors.

The authors evaluate Seq-Flow on synthetic dynamics, particle-accelerator beam spill forecasting, Burgers' equation, and real-world fluid–structure interaction data. The experimental results demonstrate improvements over several diffusion- and flow-based baselines under limited sampling budgets, with particularly strong performance on the synthetic and Beam Spill benchmarks. Additional ablation studies investigate the effects of self-rollout training and renoising on long-horizon error accumulation.

Critical strengths
  • The proposed method is conceptually simple and well-motivated. Exploiting temporal continuity between consecutive predictive distributions to reduce the computational cost of repeated probabilistic forecasting is a reasonable and potentially useful direction, particularly for online forecasting applications.
  • The ablation studies provide useful insights into the roles of self-rollout training and renoising. In particular, the observation that a model trained with only a few recursive updates can maintain stable forecasting performance over hundreds of consecutive updates is interesting.
Critical weaknesses
  • The proposed method distinguishes itself from Self-Forcing (NeurIPS 2025) by using model-generated forecasts as the source distribution for subsequent flow sampling, rather than incorporating them into the conditioning context. While this is a meaningful difference in formulation, I am not fully convinced that it constitutes a substantial technical contribution on its own.
  • While Seq-Flow demonstrates substantial improvements on the synthetic and Beam Spill benchmarks, its gains over full-trajectory flow models are relatively modest on Burgers' equation and RealPDE-FSI. For example, on RealPDE-FSI, Seq-Flow improves CRPS from 0.0029 to 0.0026 and RMSE from 0.0056 to 0.0055 compared with the three-NFE Full-trajectory Flow baseline. Furthermore, the performance–efficiency trade-off analysis in Sec. 4.2 and the error accumulation ablations in Sec. 4.3 focus primarily on the synthetic and Beam Spill benchmarks, where the proposed method exhibits its strongest improvements. Consequently, it remains unclear whether replacing the Gaussian source distribution with previous model predictions consistently provides meaningful benefits across more challenging forecasting tasks.
Questions for authors
  • While Seq-Flow demonstrates substantial improvements on the synthetic and Beam Spill datasets, the gains on Burgers' equation and RealPDE-FSI are relatively modest. Could the authors elaborate on why the improvements are less pronounced on these tasks? Additionally, could the authors extend the performance–efficiency analysis and error accumulation ablations in Sec. 4.2 and 4.3 to these datasets? This would help determine whether the benefits of previous-forecast-based flow transport generalize beyond the benchmarks on which the proposed method performs particularly well.
  • Could the authors clarify whether the comparisons in Tab. 1 include Rectified Flows? While Rectified Flow appears in Fig. 2 and 3 as a baseline, I wonder whether it is used as a baseline in the main comparison as well.
  • The experiments use root mean squared error (RMSE) as the primary pointwise prediction error metric. Could the authors additionally report normalized metrics, such as variance-scaled RMSE (VRMSE) or relative L2 error? Since the absolute magnitudes of the physical fields may vary across datasets, normalized metrics would provide a more interpretable assessment of predictive accuracy, particularly for systems whose state variables have relatively small magnitudes.
  • The reported errors for MeanFlow in Tab. 1 are notably higher than those of several other baselines. Given that MeanFlow has demonstrated strong few-step generative modeling performance in other domains (e.g., images), could the authors elaborate on why it performs relatively poorly in these forecasting tasks? Are there any particular challenges associated with adapting MeanFlow to sequential probabilistic forecasting?
Reviewer_z6JyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Seq-Flow, a method for learning an online update between successive predictive distributions. Rather than starting every forecast from Gaussian noise, it shifts a preceding forecast into the new prediction window and uses that as a source for a conditional flow, with the aim of reducing the number of required sampling steps. Additionally, to reduce mismatch/accumulation between training sources and recursively generated inference sources, self-rollout training is used. Experimental results cover a multimodal random walk, simulated beam spills, partially observed forced Burgers dynamics, and fluid-structure interaction fields, with three-evaluation updates substantially improving on beam-spill CRPS and remaining competitive on fluid tasks.

Critical strengths
  1. The core method formulation is coherent and addresses a specific failure mode of forecast reuse. A pretrained noise-to-data model need not behave well when initialized from its own earlier forecasts, and training directly on such sources appears to be a reasonable approach. The distinction from Self Forcing is meaningful, where model outputs initialize the next transport while the conditioning history continues to receive observations.
  2. The beam-spill results and long-run ablations provide compelling positive evidence for the method. Figures 3-4 demonstrate error over successive updates and contrast generated versus ground-truth sources/renoising/rollout lengths. Training on short chunks while maintaining accuracy over 412 evaluated beam-spill forecast windows appears to be a practically useful result, and the same is true for the fluid studies (although their margins are smaller).
Critical weaknesses
  1. The evaluation doesn't seem to fully establish the claimed quality or stability of the joint predictive trajectory distribution. The reported CRPS uses an L1 norm over the forecast vector (Section 4.1, p. 8), and expanding that norm gives a sum of univariate CRPS terms, which depends on coordinate marginals and cannot distinguish different temporal or spatial dependence structures with the same marginals. Analytic random-walk marginals are provided in Appendix A.1 for Wasserstein evaluation, but that doesn't explain the joint-path evaluation and RMSE doesn't fill the gap.
  2. The efficiency and self-rollout conclusions leave controlled implementation evidence to be desired, and make it challenging to assess the claimed practical advantage. The main efficiency comparison is NFE as opposed to measured end-to-end latency or total compute. It's likely that calls on a complete trajectory and calls in autoregressive generation will have have different costs, provided training requires extra EMA rollouts and repeats under the corresponding NFE budget. No wall-clock training comparison or ensemble-size accounting is provided, despite loose motivations around the 0.1ms observation interval and approximately 1ms feedback loop for beam spill prediction.
Questions for authors
  1. Does the performance advantage persist under a joint trajectory score or explicit tests of temporal/spatial dependence, including eg random-walk increment structure and multistep event probabilities? How many predictive samples are used for each reported score, and what is the variation across independent training runs (Table 1 doesn't report any confidence intervals, for instance)?
  2. What exactly does T0T_0 count in Figure 4? Under Algorithm 1, T0=1T_0=1 appears to contain only Gaussian-source training.
  3. (Relating to mentioned weakness #2) Can you provide measured latency/wallclocks times for components of the method, e.g., batch-one forecast latency, latency for the ensemble used in scoring, and training compute at matched attained quality, including overhead for initialization/EMA rollout generation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.