- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
Modeling the changes of a dynamic system from streaming observations is challenging. There have been some classical filtering approaches and there are recent advancements in flow-based generative models, which are natural extensions to use. However, transporting from noise to the target is not easy, especially in a real-time streaming setting. To address this, this paper proposes Sequential Bayesian Flow Matching (SBFM), which learns a sequence of posteriors using flow-based models. This method has better performance than some baselines on a few scientific applications presented.
Critical strengths
None.
Critical weaknesses
1. Pretraining-finetuning framework
- This approach raises some concerns about SBFM. With a standard training environment (which is described in line 257 of the main paper), I guess we can train SBFM but the performance is poor. We need an additional "pretraining" step to make the method better. Then, for example, for Figure 2-(b), aren't you just showing pretraining (rectified flow) vs pretraining + finetuning (SBFM) results? I'm not this is even a fair comparison. See questions below for further concerns.
Questions for authors
- In real-world scenarios, is often not independent of previous observations. Do you think your theoretical intuition will still hold even in this case?
- When you train SBFM, do you need additional , which is not used in the original conditional flow matching training?
- If we train a flow map after pre-training or just a naive flow matching, do you think it will outperform your proposed method within a single step?
- I'm confused about what exactly will be when converged. If every loss goes to , what would be the optimal vector field? On the other hand, in practice, since the loss will never be , I wonder whether the loss accumulation is happening from a lot of places, including separate pre-training and fine-tuning phases (+ I also want to know how the "renoise" can be justified).
- What are the sizes (number of parameters) for each baseline in the experiments? Also, did you use the same computational resources to train all of them?
- Can Warm-start Diffusion/Wamr-start Rectified Flow be seen as SBFM methods as well? I think it still follows the method's architectural intuition but just not doing the fine-tuning.
- Is "Pretrained Model" in line 363 the pretrained model for SBFM? If so, do you have results for SBFM + Diffusion as well for Table 3? Likewise, do you have results for SBFM + Rectified Flow for Table 4 and 5?
- For experiments, the paper chooses to represent the predictive distribution. This seems like a very small number to represent any distribution. Is there any reason you chose this number?
9 comments
Author Rebuttal: Regarding Questions For Authors
Here we answer the questions raised by the reviewer.
Q1: " is often not independent of previous observations."
Our formulation does not assume is independent of past observations, and the Bayesian filtering equations (1–2) require neither Markovian dynamics nor independent observations. By conditioning on the history and the previous posterior , SBFM learns a posterior-to-posterior transport regardless of how depends on previous variables.
Q2: "When you train SBFM, do you need additional x_{t-1}?"
SBFM needs samples to learn posterior-to-posterior transport, rather than the noise-to-posterior transport of conventional flow matching. These samples come from model-based data augmentation: is generated by the pretrained flow model, not drawn from an external dataset.
Q3: "If we train a flow map after pre-training or just a naive flow matching,..."
We find SBFM still outperforms naive flow fine-tuning and flow map distillation (Tables C, D).
Q4: "I'm confused about what exactly will be when converged."
The optimal velocity field at the minimum of the fine-tuning objective transports the approximate previous pretrained posterior (after renoising) to the true current posterior .
In practice, the pretrained posterior is only approximate and fine-tuning is imperfect. Errors can therefore accumulate over sequential updates, as the source distribution generated at inference drifts from the one seen in training . Re-noising reduces this mismatch by perturbing the previous posterior samples with Gaussian noise, which makes the model more robust to approximation errors. The re-noising ablation in Figure 6 of the manuscript shows that removing re-noising substantially degrades performance.
Q5: "What are the sizes for each baseline in the experiments?"
All methods share the same backbone (~2.5M, 300K, 2.5M, and 3.5M parameters for synthetic, beam-spill, Burgers', and weather, respectively).
Training-from-scratch methods use the same GPUs and number of epochs, though per-iteration cost varies by objective (e.g., MeanFlow requires Jacobian-vector products). SBFM uses the standard flow matching objective with a modified source distribution, generated by the pretrained model offline in advance, avoiding sampling overhead during fine-tuning.
Q6: "Can Warm-start Diffusion/Wamr-start Rectified Flow be seen as SBFM methods as well?"
From an inference-pipeline view, warm-start Diffusion and warm-start Rectified Flow are SBFM without fine-tuning. But the key difference is the learned transport. These methods are trained to transport Gaussian noise to the posterior , but at inference they start from the (partially) noised previous posterior . This mismatch between the source distributions at training and inference can greatly degrade sampling accuracy. SBFM instead learns the transport from to directly, so the training matches the inference-time initialization.
Q7: "Is "Pretrained Model" in line 363 the pretrained model for SBFM?"
Yes. The pretrained models in line 363 are the Diffusion and Rectified Flow models used to initialize SBFM, and both work with our framework. On Burgers’ equation, SBFM initialized from Diffusion beats the pretrained Diffusion model at the same NFE but trails SBFM initialized from Rectified Flow (Table F). We therefore reported the stronger-performing SBFM in our experiments."
Table F: Results of SBFM-Diffusion on Burgers’ equation forecasting.
| Method | NFE | L2 Energy Score↓ | RMSE↓ |
|---|---|---|---|
| Diffusion | 10 | 0.100 | 0.258 |
| Rectified Flow | 5 | 0.103 | 0.262 |
| Diffusion | 1 | 0.108 | 0.247 |
| Rectified Flow | 1 | 0.114 | 0.250 |
| SBFM-Diffusion | 1 | 0.105 | 0.242 |
| SBFM-Rectified Flow | 1 | 0.101 | 0.239 |
Q8: "For experiments, the paper chooses N=20 to represent the predictive distribution."
We thank the reviewer for the question. To check how sensitive the results are to particle count, we re-evaluated with N=100 particles. The numbers are overall robust with the N=20 results in the manuscript, and SBFM’s improvements remain (Table G).
Table G: Comparison of energy scores (lower the better) with N=20 and N=100 particles. Each entry reports N=20 -> N =100.
| Method | NFE | Beam | Burgers’ equation | Weather |
|---|---|---|---|---|
| Rectified Flow | 1 | 0.0302→0.0316 | 0.1140→0.1200 | 2.608→2.736 |
| Rectified Flow | 2 | 0.0245→0.0245 | 0.1082→0.1080 | 2.503→2.502 |
| Rectified Flow | 5 | 0.0231→0.0224 | 0.1030→0.1040 | 2.030→2.198 |
| SBFM-Rectified Flow | 1 | 0.0179→0.0165 | 0.1010→0.1010 | 2.353→2.431 |
| SBFM-Rectified Flow | 2 | 0.0170→0.0159 | 0.1016→0.1010 | 2.366→2.366 |
[1] Sabour et al. Align Your Flow: Scaling Continuous-Time Flow Map Distillation. NeurIPS 2025.
[2] Tong et al. Flow Map Distillation Without Data. CVPR 2026.
Thank you so much for the detailed rebuttal! I still have a few questions and I'll leave them below.
Questions to the answer of W1
- I'm just a bit confused here. In the "pretraining" phase, are you training the exact same flow for both rectified flow and SBFM? When you see the training curve, is it converged? If so, what do you mean by "finetuning" more for the pretrained rectified flow? Can you explain more about how these training processes work, both for pre-training and fine-tuning, including the data and training objective? I think you are generating data from the pretrained flow and using that to fine-tune it... which I don't think is really a good idea (or fair)
- How did you search for learning rates? Did you find optimal learning rates for each training method and compare them? Also is the reported number just a single seed with a fixed learning rate? If we were to validate things experimentally, I just want to see more completeness in the experimental setup so that we can really believe the numbers.
- I'm also curious to see larger NFE comparisons
- "As Tables D shows, AYF is unstable to optimize due to Jacobian computation" <- Is it impossible to run stable optimization for AYF in the Beam task? I wonder how bad the method is in terms of optimization. Can you share which implementation you follow for this?
- Same for Table D. How confident are you in terms of training to optimality for the baselines? Since the number seems to be a single-seed training (or evaluation) value, it's not so convincing yet (I could be wrong so please provide more details if possible!).
- Table E seems very interesting and good. However, (as in many on-policy methods) it seems like there could be a lot of difficulties in the initial phase of training. Were there any issues during training? Table E shows the synthetic case but does it also work without any error in more realistic tasks as well? Since the numbers look much stronger than the original proposed method, I'm very curious about this approach.
Questions to the answer of Qs
- Q4: I think it's not clear yet. Can you give a clear formulation for the optimal such that when the loss is exactly 0? Also, you mentioned "Re-noising reduces this mismatch" but how exactly? Can you provide a formulation/theory for this? I don't quite get it. Define the exact mismatch and prove how re-noising can reduce it.
- Q8: Thanks for the additional experiment! Can you also report the time it takes to sample for each method as well for different N? Also, can you also run different metrics so that you can capture better divergences between the distributions? Still seems like a small number of particles to explain the whole distribution so I'd rather want to see other metrics as well.
Overall, once again, thank you so much for the additional experiments and efforts for the rebuttal. I am willing to adjust my score and confidence based on the answers to the questions above.
Author Rebuttal: Regarding Pretraining-finetuning Framework
We appreciate the reviewer's time and efforts for providing the comments. Here is our response.
W1 (Pretraining-Finetuning Framework)
We thank the reviewer for raising this concern. While we acknowledge that SBFM requires additional training effort, its fine-tuning objective primarily leverages bootstrapped data generated by the pretrained model to learn posterior-to-posterior flows, with the goal of improving inference-time sampling efficiency. Importantly, the improvement is not merely due to an extra finetuning compute alone, but our proposed SBFM objective.
To demonstrate this, in the following we conduct controlled experiments in which the same pretrained flow is further fine-tuned using conventional flow matching or flow map distillation under matched training computational budgets. The results show that SBFM consistently outperforms these alternatives at low NFE, highlighting the benefit of the proposed posterior-to-posterior flow learning objective. Furthermore, we show that the two-stage training procedure of SBFM can be unified into a single training stage by generating self-rollout data on the fly, eliminating the need for separate pretraining while preserving SBFM's superior performance.
1. Pretraining + conventional flow fine-tuning yields no further improvement
To separate the benefit of our formulation from the effect of extra fine-tuning, we continued fine-tuning the pretrained Rectified Flow model with standard noise-to-posterior flow matching, using the same data and compute budget as SBFM. This corresponds to setting the re-noising level to 1 in our pipeline. Standard fine-tuning gives only a marginal improvement over the pretrained model, while SBFM improves substantially (see Table C below). It implies the gains therefore come from the posterior-to-posterior objective rather than from additional training or computation.
Table C: Comparing SBFM and standard noise-to-posterior flow finetuning on a controlled synthetic state estimation task (see Appendix C.1 for concrete setup). The latter is implemented by SBFM with renoise level = 1 so that the source distribution becomes purely Gaussian.
2. Pretraining + flow map distillation underperforms SBFM
We also compare SBFM with two flow map distillation methods that, like SBFM, use a pretraining–fine-tuning pipeline: (a) AYF [1], closely related to MeanFlow, using its Eulerian Map Distillation variant; and (b) FreeFlow [2], a data-free method that mitigates teacher–data mismatch. Both compress noise-to-target transport into a single network evaluation, whereas SBFM accelerates inference by learning posterior-to-posterior transport. For a controlled comparison, both students are initialized from SBFM’s pretrained velocity networks and fine-tuned for the same number of epochs. As Tables D shows, AYF is unstable to optimize due to Jacobian computation, while FreeFlow improves few-step sampling but still underperforms SBFM.
Table D: Comparison with flow map distillation approaches (AYF and FreeFlow) and conventional Rectified Flow on beam spill and Burgers’ equation forecasting.
3. A single-stage SBFM training from scratch
We further show that we can trarin SBFM from scratch by a self-rollout training strategy. Specifically, instead of pretraining a conventional rectified flow model before fine-tuning SBFM, we initialize both models simultaneously and train them jointly. At each training step, the conventional rectified flow model is updated using the standard flow matching objective, while its current predictions are used to generate bootstrapped data on the fly for training SBFM with the proposed posterior-to-posterior flow objective. As shown in Table E, SBFM trained from scratch using this self-rollout strategy, with the same training dataset as the pretrained flow model, preserves the superior performance–efficiency trade-off.
Table E: W1 distance on synthetic random-walk dataset.